Acoustic signal transmission with insertion signal for machine control

ABSTRACT

The acoustic signal transmission includes electrically synthesizing an audible sound signal and an insertion signal to generate a synthesized sound electrical signal at the sending side, transmitting the synthesized sound electrical signal, and extracting the insertion signal form the synthesized sound electrical signal at the receiving side. The extracted signal may be used for controlling machines or for transmitting information. The acoustic signal represented by the synthesized sound electrical signal can be heard by human ears in the same way as the audible sound signal, while the synthesized insertion signal cannot be detected by human ears. The synthesized sound electrical signal can be transmitted as a sound wave in air space, as electrical signals through a transmission line or as radio signals such as infrared and electromagnetic waves. The synthesized sounds may also be transported or distributed by recording on recording media such as compact disc and DVD.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an acoustic transmission method and anapparatus for transmitting signals using sound waves as the transportmedium.

This application is based on patent application Nos. Hei 11-329914, Hei11-356827. Hei 11-363811, Hei 11-366345, Hei 11-366346, 2000-136716,2000-136717, 2000-248800 filed in Japan, the contents of which areincorporated herein by reference.

2. Description of the Related Art

Conventionally, when transmitting the sound that can be heard by humansor animals (audible sound signals) and associated information signalsconcurrently to some machine, the audible signal and the associatedinformation signal are sent through separate channels.

For example, audible signal and information signal are sent aselectrical signal to modulate the carrier wave, and are sent to thereceiver side through electromagnetic waves and other media such ascables, and are used after demodulation by the receiver side. In suchmethods, it is necessary to provide respective transceivers for thepurpose of sending an audible sound signal separately from anothersignal different than the audible sound signal.

That is, a total of more than two apparatuses are necessary, andcomplexity in the overall apparatus configuration is unavoidable. Also,apart from this problem, although there are methods of signaltransmission based on ultrasonic waves as carrier waves, becauseultrasonic sounds cannot be heard by human ears, it is necessary toconvert to an audible sound at the receiver side, and the method cannotbe used for the above purpose.

As a technology similar to the technologies described above fortransmitting sound (audible sound signals) that can be heard by humansor animals simultaneously with associated information signals, isdisclosed in a Japanese Unexamined Patent Application, FirstPublication, No. Hei 8-37511 “Interactive broadcasting system andreceiving system”, Publication date, 6 Feb. 1996 (Reference 1), forexample. This method is based on simply superimposing signals accordingto the DTMF (Dual Tone Multiple Frequency) format on the audible soundsignals.

However, according to this method, sounds that are not needed to beheard and are not of any interest to humans can be heard clearly asbackground noise. Therefore, it is very disturbing, and possibilitiesexist of misunderstanding the intended meaning of the original soundsrepresented by the audible sound signals.

Therefore, such methods cannot be said to be suitable as sound-basedinformation transfer means.

SUMMARY OF THE INVENTION

Therefore, an object of the present invention is to provide means toeasily transmit sounds (audible sound signals) that can be heard byhumans and animals and to simultaneously transmit associated informationsignals. Also, the present invention has an object to provide varioussystems based on the information transmission means.

According to the present invention, the object has been achieved in amethod for transmitting acoustic signals comprising: a synthesizing stepfor synthesizing an audible sound signal and another signal differentthan the audible sound signal to generate a synthesized sound electricalsignal; an acoustic signal outputting step for converting thesynthesized sound electrical signal to an acoustic signal and outputtingthe acoustic sound externally; a transmitting step for transmitting thesynthesized sound electrical signal; and an extracting step forextracting said another signal from the synthesized sound electricalsignal that has been transmitted.

In other words, in the present invention, the audible sound signal andanother signal different than the audible sound signal are synthesizedelectrically, using a data hiding technique, for example, under acondition that the auditory presence of the signal cannot be detected byhuman ears, and this is emitted from a sound generation apparatus(speaker for example) to carry out signal output.

In this case, synthesizing means may use existing data hiding techniquefor embedding ID information and the like in voice sound information.The data hiding technique is reported in “Special Issue, “ElectronicWatermark” protects multimedia age, Nikkei Electronics, 24 Feb. 1997,(no. 683), pp. 99–124 (Reference 2), “Article, Data hiding technique tosupport electronic watermark (part one)”, Nikkei Electronics, 24 Feb.1997, (no. 683), pp. 149–162 (Reference 3), “Article, Data hidingtechnique to support electronic watermark (part two)”, NikkeiElectronics, 10 Mar. 1997, (no. 684), pp. 153–168 (Reference 4).

At the receiving side, the synthesized sound is collected by amicrophone, and from a converted electrical signal, signal is extracted.Therefore, persons near the speaker, unaware of the auditory presence ofthe signal, is able to listen to the voice sound.

Using such a method, it is possible to easily transmit voice soundsignals and another signal different than the voice sound signal using asimple apparatus. Especially, when transmitting signals from the sendingside by voice sound that can transmit through air, the apparatus on thereceiving side is only a voice sound generation means representedtypically by a speaker, and the receiving side is a sound collectionapparatus represented typically by a microphone, and therefore, radiotransceiver or wired transceiver is not necessary so that an advantageis that the structure of the overall system is simple and veryeconomical.

Also, accordingly, once the synthesized sounds are recorded, soundreproduction apparatuses in all kinds of apparatuses, such as personalcomputer, karaoke player, radio, television, tape deck, video deck, MDplayer, CD player, DVD player, analogue record player, DAT deck, MP 3player can be utilized directly as a sending apparatus. Further, therecorded synthetic sounds, when they are recorded on a sound recordingmedium, can be distributed.

Also, recorded synthetic sounds are able to be transmitted directly asdirect data through transmission networks such as the Internet,telephone networks and broadcasting networks. Also, synthetic sounds caneasily accumulated, modified, processed, analyzed and stored. Also,using one medium called sound, two pieces of information can be sentsimultaneously.

Also, according to the present invention, applying such informationtransmission means, it becomes possible to provide means fortransmitting acoustic information, such as music, and control signals tocontrol motions and sound generation of a robot to match the voice soundinformation simultaneously and easily.

Also, according to the present invention, applying such informationtransmission means, voice sounds broadcast by radio and the like andsuch information as traffic information or sightseeing information orcommercial information that can be used in car navigation purposessimultaneously. Also, in order to realize this, without having toinstall FM multiplexing apparatus and the like at the broadcastingstation side, information can be embedded directly in the voice signalsthemselves, so that the system can be constructed very economically.

Also, according to the present invention, utilizing the informationtransmission means such as the one described above, voice soundsbroadcast through a radio and the like and URL information from the siteon the Internet can be simultaneously transmitted, and by using the URL,it becomes possible to rapidly access the site. The receiver side,accordingly, does not need to copy the URL or to take notes or tomemorize. Also, without altering the facility of conventional radio,information via the Internet can be accumulated in a car navigationapparatus. Also, because the access is made easier, the sponsor canexpect to have an increased number of access to its home page. Also,promotion that links commercial broadcasting and home page can berealized.

Also, according to the present invention, utilizing informationtransmission means such as the one described above, by transmittingvoice sounds obtainable from TV receiver to a robot, through TVbroadcasting and the like, it enables to control robot operation andsound generation. Also, it enables to provide feedback from the viewerthrough the robot, interactive television broadcasting can be realized.

Also, according to the present invention, utilizing informationtransmission means such as the one described above, using only voicetransmission means, it enables to send signals to control the robot frompersonal computers and the like. Accordingly, the robotic system can besimplified and cost lowered. Also, because there is no need for adedicated line in the computer system for controlling the robot, anadvantage is that, while using the robot, other devices such as scanner,terminal adapter, printer can simultaneously be used.

Also, according to the present invention, utilizing informationtransmission means such as the one described above, it enables tobroadcast data relating to coupons for sales promotion along withcommercial broadcast, and to extract the coupon data at the receiverside. Then, it enables to accumulate the coupon data in personalcomputer and the like on the receiver side, sending the coupon data to aweb server through the Internet and the like, checking the coupon datareceived at the web server side and accumulate the coupon data for eachreceiver. Then, it enables to award special business incentive, such asprice discount on commercial goods, to the receiver according toaccumulated coupon data. Accordingly, the receiver side can increase theenjoyment of actively listening to the broadcast commercial, while thesponsor can benefit from effective advertising and increased potentialsales.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are diagrams to show a first embodiment of the presentinvention.

FIG. 2 is a diagram to show a method of utilizing a frequency band inthe first embodiment.

FIG. 3 is a diagram to show an example of a synthesizing apparatus inthe first embodiment.

FIGS. 4A and 4B are diagrams to show the operations of the synthesizingapparatus in the first embodiment.

FIG. 5 is a diagram to show an example of the structure of an extractionapparatus in the first embodiment.

FIG. 6 is a diagram to show a second embodiment of the presentinvention.

FIG. 7 is a diagram to show a method of utilizing a frequency band inthe second embodiment.

FIG. 8 is a diagram to show the structure of the synthesizing apparatusin the second embodiment.

FIGS. 9A and 9B are diagrams to show the operations of the synthesizingapparatus in the second embodiment.

FIG. 10 is a diagram to show an example of the structure of anextraction apparatus in the second embodiment.

FIGS. 11A and 11B are diagrams to show a third embodiment of the presentinvention.

FIG. 12 a diagram to the third embodiment of the present invention.

FIG. 13 is a diagram to show an example of the structure of asynthesizing apparatus in the third embodiment.

FIG. 14 is a diagram to show an example of the structure of a machinecontrolling section in the third embodiment.

FIG. 15 is a diagram to show an example of the structure of asynthesizing section in the third embodiment.

FIG. 16 is a diagram to show an example of the correlation of audiosignals and operation signals and machine operation in the thirdembodiment.

FIG. 17 is a diagram to show an example of the correlation of audiosignals and operation signals and machine operation in the thirdembodiment.

FIG. 18 is a diagram to show an example of the structure of asynthesizing section in the third embodiment.

FIG. 19 is a diagram to an example of the structure of a synthesizingsection in the third embodiment.

FIG. 20 is a diagram to show the basic structure of a fourth embodimentof the present invention.

FIG. 21 is a diagram to an example of the structure of a synthesizingsection in the fourth embodiment.

FIGS. 22A and 22B are diagrams to show examples of the relationship ofthe input signal and switching operation in the fourth embodiment.

FIG. 23 is a diagram to show an example of the structure of anextraction section in the fourth embodiment.

FIG. 24 is a diagram to show a second example of the structure of thesynthesizing apparatus in the fourth embodiment.

FIG. 25 is a diagram to show a second example of the structure of theextraction apparatus in the fourth embodiment.

FIGS. 26A and 26B are diagrams to show the basic structure of a fourthembodiment of the present invention.

FIG. 27 is a diagram to show an example of the chronologicalrelationship of music signals and dance operation codes in a fifthembodiment.

FIG. 28 is a diagram to show the attitude of a dance operation of arobot and the corresponding dance operation codes.

FIG. 29 is a diagram to show an example of the structure of anextraction section in the fifth embodiment.

FIG. 30 is a flowchart to show the processing steps in the synthesizingapparatus in the fifth embodiment.

FIG. 31 is a diagram to show an example of the structure on the receiverside.

FIGS. 32A and 32B are block diagrams to show the basic structure inEmbodiment 6.

FIG. 33 is a block diagram of an example of the structure in theautomobile side in the Embodiment 6.

FIGS. 34A and 34B are block diagrams of an example of the structure in aEmbodiment 7 of the present invention.

FIG. 35 is a block diagram of another example of the embodiment inEmbodiment 7.

FIG. 36 is a diagram to show an example in Embodiment 7.

FIG. 37 is a block diagram of a functional structure of a robot inEmbodiment 7.

FIG. 38 is a diagram to show an example of the correlation data for theoperational command for the robot in Embodiment 7.

FIG. 39 is a diagram to show an example of the correlation data for theconversational command for the robot in Embodiment 7.

FIG. 40 is a diagram to show another example in Embodiment 7.

FIG. 41 is a block diagram to show a functional structure of a robot inEmbodiment 7.

FIG. 42 is a block diagram to show an integrated system of communicationand broadcasting in Embodiment 7.

FIG. 43 is a diagram to show an example of the correlation data for theoperational command for the robot in Embodiment 7.

FIG. 44 is a diagram to show an example of the correlation data for theconversational command for the robot in Embodiment 7.

FIG. 45 is a diagram of the overall structure of the robot in Embodiment9 of the present invention.

FIG. 46 is a block diagram of the functions in a personal computer sideof a robot in Embodiment 9.

FIG. 47 is a block diagram of the functions in the robot side of therobotic system in Embodiment 9.

FIG. 48 is a diagram of the details of the robot control section inEmbodiment 9.

FIG. 49 is a diagram to show a relationship of signals in the personalcomputer side in Embodiment 9.

FIG. 50 is a block diagram of the functions in a variation of therobotic system shown in Embodiment 9.

FIG. 51 is a flowchart of the synthesized sound electrical signalgeneration process in Embodiment 9.

FIG. 52 is a block diagram of the functions on the personal computerside in Embodiment 10.

FIG. 53 is a block diagram of the functions on the robot side of therobotic system.

FIG. 54 is a diagram to show the relationship of each signal on thepersonal computer side in Embodiment 10.

FIG. 55 is a diagram of the overall structure of the robot in Embodiment11 of the present invention.

FIG. 56 is a diagram of the overall structure of the robot in Embodiment12 of the present invention in Embodiment 12.

FIG. 57 is a diagram to explain a method of manufacturing a CDcontaining synthesized sound electrical signals in Embodiment 12.

FIG. 58 is a block diagram of the function on the robot side in arobotic system in Embodiment 13.

FIG. 59 is a block diagram of the function on the robot side inEmbodiment 13.

FIG. 60 is a block diagram of an example of broadcasting commercialmessages.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following embodiments do not restrict the interpretation of theclaims relating to the present invention, and the combination of all thefeatures explained in the embodiments is not always being indispensablemeans of solving the problem.

In the following, preferred embodiment of the present invention will beexplained in detail with reference to the drawings.

Embodiment 1

First, examples in Embodiment 1 will be explained.

FIGS. 1A and 1B are diagrams to show Embodiment 1 of the presentinvention, and numeric symbol 1A refers to the sending side and 1Brefers to the receiving side. In FIG. 1A, numeric symbol 1 representsaudible sound signal, 2 an insertion signal, which is another signaldifferent than the audible sound signal 1, and 3 a synthesized soundelectrical signal.

Although the following explanations relate to a case of using a digitalsignal for signal 2, but even if the signal 2 is an analogue signal,once the signal has been converted to digital information by processingthe signal through an A/D converter, it is obvious that it can behandled in the same manner as digital signals.

Also, 4 represents a synthesizing apparatus, 5 an amplifier (in FIG. 1A,it is recited as AMP), 6 a speaker, 7 a synthesized sound, 8 a person.In FIG. 1B, numeric symbol 9 represents a microphone, 10 an extractionapparatus, and other numeric symbols are the same as those in FIG. 1A.

An outline of the flow of the signal in FIGS. 1A, 1B will be explained.First, on the sending side, audible sound signal 1 and signal 2 to betransmitted other than the audible sound signal 1 are electricallysynthesized using, for example, a data hiding technique, under acondition such that the auditory presence of signal 2 cannot be detectedby the person, and, after passing through the amplifier 5, a synthesizedsound 7 is emitted from the speaker 6 into the air space.

A person who is nearby can hear this synthesized sound by own ears. Onthe other hand, on the receiving side, the emitted sound is collected bythe microphone 9, and, after passing through the amplifier 5, isconverted to the synthesized sound electrical signal. Then, the signal 2is extracted from the synthesized sound electrical signal by theextraction apparatus 10.

FIG. 2 is a diagram to show a method of utilizing the frequency band(band-elimination method) in Embodiment 1. FIG. 3 is a diagram to showan example of the structure of the synthesizing apparatus in FIG. 1A,and numeric symbols 1˜3 are the same as those in FIG. 1A, 20 representsthe synchronizing signal, 21 a switching circuit, 22B a band-eliminationfilter 0, 22A a band-elimination filter 1 (band-elimination filter isabbreviated as BEF in the diagram), 23 a machine signal-switching signalconverter, and 24 a synthesizer.

FIGS. 4A, 4B are diagrams to explain the operation of the synthesizingapparatus shown in FIG. 3. FIG. 4A shows the properties of theband-elimination filter, and numeric symbol 25 show the properties ofthe band-elimination filter 0 and those of the band-elimination filter1, respectively. Also, FIG. 4B shows the relationship of the waveform ofthe synthesized sound electrical signal and machine operation signal(signal 2) and the changeover switch 21.

FIG. 5 is a diagram to show an example of the structure of theextraction apparatus shown in FIG. 1B, and numeric symbols 2, 3, 10 arethe same as those in FIG. 1B, 20 represents a synchronizing signal, 27 afrequency analyzer (abbreviated as FET in the diagram), 28 an energydetector, and 29 a machine signal converter.

Embodiment 1 will be explained in the following with reference to thediagrams. The synthesizer 4, as shown in FIG. 3, is comprised by aband-elimination filters 22A, 22B to eliminate a certain band fromsignal 1 depending on the value of the binary signal 2, and when sendinga code “1”, the sound energy is eliminated in a particular centralfrequency A by the band-elimination filter 22A (BEF1).

Similarly, when sending a code “0”, the sound energy is eliminated in aparticular central frequency B that is different than the centralfrequency A by the band-elimination filter 22B (BEF0). Humans arevirtually unable to distinguish an audible sound uniquely lacking aspecific frequency only from the original sound in their normal auditorystate in daily living space.

Suppose that, even when such existence is able to be detectedphysically, humans are not able to meaningfully recognize itssignificance. Transmission of signal 2 is carried out as follows.

(Step 1) the machine-signal-switching-signal converter drives theswitching circuit 21, in accordance with the polarity of signal 2, whileswitching the band-elimination filter for signal 1 according to “1” and“0” to generate a sound signal from which a specific superposition hasbeen eliminated from signal 1.

(Step 2) after synthesizing the filtered sound signal in thesynthesizing apparatus 24, it is amplified in the amplifier 5 and isemitted into the air space through the speaker 6.

Decoding at the receiver side is carried out by receiving the soundsignal emitted into the air space in the microphone 9, and afteramplifying in the amplifier 5, signal 2 is extracted in the extractionapparatus 10. In the extraction apparatus 10, the synthesized signal 3input is analyzed in the frequency analyzer 27, the energy is detectedin the energy detector 28, and the missing frequency component isextracted, and this is converted in the machine signal converter 29 intomachine signals “1” or “0” so as to output as signal 2.

Decoding at the receiver side can also be realized by anotherconfiguration different than the extraction apparatus 10. That is, thedecoder is comprised by two band-pass filters, and by making theirrespective center frequencies equal to those of the band-eliminationfilters 22A, 22B of the synthesizing apparatus 4, the signal 2 can bedecoded on the basis of the magnitude of the output from each band-passfilter.

Specific receiving steps for signal 2 can be summarized as follows.

(Step 1) detect a sound signal propagating through the air space bymicrophone;

-   -   (Step 2) amplify the signal detected by the microphone;    -   (Step 3) obtain an output signal from a band-pass filter by        passing the amplified signal therethrough;    -   (Step 4) extract signal 2 by carrying out decoding according to        comparison of the output signal with an appropriate threshold        value.

Embodiment 2

FIG. 6 shows Embodiment 2 of the present invention, which shows both thesending side and the receiving side. In the diagram, numeric symbol 1represents an audible sound signal; 2 a signal different than theaudible sound signal; 3 a synthesized sound electrical signal; 4 b asynthesizing apparatus; 6 a speaker; 7 a synthesized sound; 8 a person;10 an extraction apparatus; 11 a sending apparatus; 12 a signaltransmission path; and 13 a receiving apparatus.

An outline of the flow of the signal in FIG. 6 will be explained. Afeature of the present embodiment is that, as illustrated in the diagramby a dotted line, there is a means for transmitting (signal transmissionpath 12) the synthesized sound 7 from the sending side to the receivingside directly without propagating through the air space. In this case,an advantage is that, because acoustic noise in the air space is notmixed in the sending signal, reliability of signal transmission isincreased. Also, at the same time, the synthesized sound is emitted intothe air space and acts directly on the ear of a person.

FIG. 7 shows a method of frequency-band utilization in Embodiment 2. Inthis case, transmission method based on sine wave is adopted. Thesynthesizing apparatus 4 b in FIG. 6 is an apparatus to convert a binarysignal into an acoustic electrical signal, and is comprised by anoscillator, so that, when sending a “1” code, a high frequency C havinga specific frequency is generated, and similarly, when sending a “0”code, a high frequency D having a specific frequency different than thehigh frequency C is sent.

Humans are virtually unable to distinguish an audible sound uniquelylacking a specific frequency from the original sound in daily living airspace in the normal auditory state. Even if such an existence can bedetected physically, humans are not able to meaningfully recognize thesignificance of signal 2.

FIG. 8 is a diagram to show an example of the synthesizing apparatus inFIG. 6, and the numeric symbols 1˜3 are the same as those in FIG. 1A,and 4 b represents a synthesizing apparatus; 20 a synchronizing signal;21 a switching circuit; 23 a machine signal-switching signal converter;24 a multiplexer; 31A an oscillator 0; and 31B an oscillator 1.

FIGS. 9A, 9B are diagrams to explain the operation of the multiplexershown in FIG. 6. FIG. 9A shows the frequency properties of theoscillator, and numeric symbol 32 refer to the frequency properties ofan oscillator 31A, 33 to those of an oscillator 31B. FIG. 9B shows arelationship of the waveform of the synthesized sound electrical signaland the machine signal (signal 2) in relation to switching by switchingcircuit 21.

FIG. 10 is a diagram to show an example of the structure of theextraction apparatus shown in FIG. 6, and numeric symbols 2, 3, 10 arethe same as those in FIG. 1B, and 27 represents a frequency analyzer(abbreviated as FFT in the diagram); 28 an energy detector; 29 a machinesignal converter. Embodiment 2 will be explained below with reference tothese diagrams.

The synthesizing apparatus 4 b as shown in FIG. 8 is provided withoscillators 31A, 31B for impressing different frequencies on signal 1,depending on the value of the binary signal 2, so that when sending a“1” code, a high frequency C having a specific frequency is generated,and similarly, when sending a “0” code, a high frequency D havinganother specific frequency is sent.

Sending of signals is performed according to the following steps.

(Step 1) operate oscillators 31A, 31B for generating sine waves ofspecific frequencies (high frequency C, D) to correspond with “1” and“2” for signal 2;

-   -   (Step 2) the machine signal-switching signal converter 23        operates the switching circuit 21 according to the polarity of        signal 2 so as to add the selected specific sine wave and signal        1 in the synthesizer 24;    -   (Step 3) the signal obtained in step 2 above is amplified in the        amplifier (not shown), and the signal is emitted into the air        space through the speaker 6 and, concurrently, the sending        apparatus sends the signal to the transmission path 12.

In the receiving side, the signal is received by way of the transmissionpath 12. The extraction apparatus 10 decodes the synthesized soundelectrical signal again into a code signal. At this time, the extractionapparatus 10 analyzes the synthesized signal 3 input therein in thefrequency analyzer 27, detects its energy in the energy detector 28, andthe impressed frequency component is extracted, and this is convertedinto machine signals “1” and “0” in the machine signal-switching signalconverter, and is output as signal 2.

Decoding at the receiver side can also be realized by anotherconfiguration different than the extraction apparatus 10. That is, thedecoder is comprised by two band-pass filters, and by making theirrespective center frequencies equal to those of the oscillators 31A, 31Bof the synthesizing apparatus 4 b, the signal 2 can be decoded on thebasis of the magnitude of the frequency components contained in signal1.

Specific receiving steps for signal 2 can be summarized as follows.

(Step 1) receive an electrically synthesize sound signal propagatingthrough the signal transmission path 12;

-   -   (Step 2) detect the specific frequency component in the received        electrically synthesize sound signal;    -   (Step 3) discriminate and output its polarity (binary value)        according to the frequency component of the detected signal;    -   (Step 4) extract signal 2 by carrying out decoding according to        comparison of the output signal with an appropriate threshold        value.

Accordingly, two examples of the structure of sending side and fourexamples of the structure of receiving side are illustrated inEmbodiments 1 and 2, but the data hiding methods are not limited tothese mentioned, and similar acoustic transmission methods may berealized by using various other methods of data hiding algorithm.

In the above explanations, specific examples of audible sound signal 1include natural language audio sounds, machine synthesized sounds,musical tones, warning sounds, natural sounds in the natural world,animal sounds and noises. Also, signal 2 represents data that areexpressed in digital or analogue forms, and specific examples includenatural language sounds, music, warning sounds, noises, MIDI (musicalinstrument digital interface) data, acoustic signals such as MP3,multimedia signals such as text, image and video signals, and sensorysignals such as touch, smell and force.

Also, the media for transmitting synthesized sound electrical signals inEmbodiment 2 include, specifically, sound waves, ultrasonic waves,electrical waves, and electromagnetic waves such as infrared, visibleand ultra-violet radiations. Also, sending and receiving means mayconsider using broadcasting such as television and radio, CATV (cabletelevision), police radio, public phones, portable phones, PHS (PersonalHandy-phone System), Internet and LAN (Local Area Network).

Embodiment 3

Embodiment 3 will be explained in the following. In Embodiment 3, avoice response machine that can respond to audio control signals isrealized by applying the present invention to machine control.

Media for enabling wireless remote control, without connecting a machineto be controlled and a control apparatus directly by means of conductivewires, such as electrical waves, infrared radiation, light and soundwaves, have long been known and used. Of these, a representative methodusing sound waves in the audible frequency band is a based on commandscomprised by synthesized sound signals that can be processed readily bymachines.

Such methods based on commands by synthesized sound signals that can beeasily understood by machines have the advantage that recognition rateis higher than a method based on natural language, but the commands aredifficult to be understood directly by humans.

For this reason, in a communication air space in which machines andhumans coexist, a method using two information channels has been adoptedwhen mutual understanding of each other's intentions is necessary.

That is, the natural language sounds are sent to humans by way of aspeaker, and separately but concurrently, signals to control the machinecorresponding to the information are sent to the machine by means suchas electrical waves. In this case, it is necessary to have sending andreceiving apparatuses for both natural language sounds and controlsignals by electrical waves and the like, resulting in a problem thatthe overall structure for the apparatus becomes complex.

Also, this method is basically powerless under situations such asunderwater and hospitals where electrical waves cannot be used. However,as explained below, Embodiment 3 of the present invention provides meansfor readily communicating mutual intentions in the communication airspace shared by machines and humans.

FIG. 11A shows the configuration on the sending side in Embodiment 3,and FIG. 11B shows, similarly, that on the receiving side. Numericsymbols 1˜8 in FIG. 11A are the same as those in FIG. 1A. Also, in FIG.11B, numeric symbol 211 represents a machine control section, 212 amachine operation command signal, 213 a machine, and other numericsymbols are the same as those in 1B.

Features of the signal flow in Embodiment 3 are as follows. The signal 2extracted in FIG. 11B is input in the machine control section 211, andis converted in the machine control section 211 into a machine operationcommand signal 212 to actually control the operation of the machine 213.The machine 213 operates in accordance with the contents of the machineoperation command signal 212.

The structures for the synthesizing apparatus in FIG. 11A and theextraction apparatus FIG. 11B can be used in Embodiments 1 and 2, forexample.

FIG. 12 is a diagram for using a transmission path instead of extractinga signal for machine control based on the synthesized sound 7 that ispropagated through the air space. As shown in FIG. 12, the electricalsignal output by the synthesizing apparatus 4 is transmitted by way ofthe transmission path 224, and is input in the amplifier 5 in thereceiving side. Other operations are the same as those in FIG. 11A and11B.

FIG. 13 is a diagram to show an example of the structure of thesynthesizing apparatus shown in FIG. 11A. In this example, characterstrings in the natural language is used as signal 2, and numeric symbols1˜4 are the same as those in FIG. 11A.

Also, the numeric symbol 214 represents a machine signal/soundcorrelation section, 215 a voice data read section, 216 a voice dataROM, 217 an A/D conversion section, 218 an ASCII character code table,219 an ASCII code converter, 221 a ROM addressing signal, 222 PCM (PulseCode Modulation) voice data, and 223 an ASCII code signal. The voicedata ROM 216 stores PCM voice data such as “GO”,” “STOP”, “RIGHT”,“LEFT” and others.

In FIG. 13, signal 2 is a character string of a natural language. Forexample, suppose that a signal meaning “GO” is input as signal 2. TheASCII code converter 219 outputs an ASCII code 223 to correspond with“GO” by referencing the ASCII code character code table 218.

This ASCII code 223 is input in the machine operation signal/voicecorrelation section 214, and the machine operation signal/voicecorrelation section 214 outputs a ROM addressing signal 221. The ROMaddressing signal 221 is a data showing the leading address (of “GO” inthis case) of the corresponding PCM voice data in the ROM voice data222.

The voice data read section 215 receives this and reads the PCM voicedata “GO” from the ROM voice data 216 and outputs the data. The PCMvoice data “GO” is converted to analogue data in the D/A conversionsection 217, and becomes an audible sound signal 1. The audible soundsignal 1 and the ASCII code signal 223 are input in the synthesizingapparatus 4 to be synthesized, and a synthesized sound electrical signal3 is output.

FIG. 14 is a diagram to show an example of the structure of the machinecontrol section, and the numeric symbols 2, 3, 10 and 211˜213 are thesame as those in FIG. 11, and 225 represents an operation signal/machineoperation command signal table. In FIG. 14, the synthesized sound signal3 collected by the microphone and amplified is input in the extractionapparatus 10, and signal 2 (operation signal) is extracted by theextraction apparatus 10.

The signal 2 in this case is a signal comprised by a natural languagecharacter string corresponding to “GO”. This signal is input in themachine control section 211, and the machine control section 211 outputsa machine operation command signal 212 corresponding to the characterstring “GO” by referencing the operation signal/machine operationcommand table 225. The machine 213 operates in accordance with themachine operation command signal.

FIG. 15 is a diagram to show another example the synthesizing apparatus,and shows a case of using a binary signal for signal 2. The numericsymbols in the diagram are the same as those shown in FIG. 13 explainedearlier.

In FIG. 15, signal 2 is a binary signal, and is to be used directly as amachine operation signal. In this case, it is assumed, for example, thata machine operation signal to correspond with “COOL” has been input. Themachine operation signal 2 is input in the machine operationsignal/voice correlation section 214, and the machine operationsignal/voice correlation section 214 outputs a ROM addressing signal221.

The ROM addressing signal 221 is a data showing the leading address (of“COOL” in this case) of the corresponding PCM voice data in the ROMvoice data 222. The voice data read section 215 receives this and readsthe PCM voice data “COOL” from the ROM voice data 216 and outputs thedata.

The PCM voice data “COOL” is converted to analogue data in the D/Aconversion section 217, and becomes an audible sound signal 1. Theaudible sound signal 1 and signal 2 (machine operation signal) are inputin the synthesizing apparatus 4 to be synthesized, and a synthesizedsound electrical signal 3 is output.

The operation at the receiving side is the same as that explainedearlier based on FIG. 14.

As explained above, in Embodiment 3, the voice signal 1 and thecorrespond-ing machine operation signal (signal 2) are first synthesizedat the sending side into one synthesized sound electrical signal byusing the data hiding technique, and is emitted into the air space as asynthetic sound through the speaker by way of the amplifier.

The meaning of the sound signal can be readily understood by a personwho is in the vicinity by listening with own ears. In the meantime, atthe receiver side, the broadcast sound is collected by the microphone,and is converted to a synthesized sound electrical signal through theamplifier. From this synthesized sound electrical signal, the machineoperation signal (signal 2) is extracted by the extraction apparatus.

Next, in the machine controller, the input machine operation controlsignal 2 is interpreted, and a corresponding machine control value isgenerated. The machine performs a specific operation according to thecommand value.

FIG. 16 is a diagram to explain a first example of the correlation ofthe voice signal and the operation signal and the machine operation, andshows a case of controlling a motion machine, such as a forklift, forexample. The diagram in the column for the movement pattern in FIG. 16shows a top view of the motion machine, and the hatched portionrepresents a vehicle body and the four wheels attached to the vehiclebody.

The correlation of the voice signal and machine operation signal incontrolling the motion machine is as shown in the columns in FIG. 16.For example, if it is desired to move the machine forward, “GO” ispronounced as the voice signal 1. And, as the machine operation signalto correspond with this command, a natural language character string “G,O” or, when using binary signals, (“0”, “1”) may be set.

In this case, the natural language character string can be expressed asin the example of configuration of the synthesizing apparatus shown inFIG. 13, alphabetical ASCII codes (8-bit information) corresponding tothe natural language character string. In this way, one character in theASCII code can be expressed by 8-bits. For example, “GO” becomes a16-bit code consisting of “01000111” and “01001111”.

The synthesized sound is emitted from the speaker and is collected bythe microphone at the receiver side. Concurrently, a person in thevicinity hears the machine sound “GO”, and can readily understand thefact that a command has been issued from the sending apparatus to themotion machine to move the machine forward. At the receiving side, thefollowing operations are performed.

That is, the operation signal (signal 2) is extracted from thesynthesized sound electrical signal received by the extraction apparatus10. In the case of the above example, either the bit strings “01000111”and “01001111” expressing the character string “GO” or a binary signal(“0”, “1”) shown in FIG. 16 is extracted. The meaning of the operationsignal (signal 2) thus extracted is interpreted by the machine controlsection.

Then, the operations signal is converted by the machine control section211 into machine operation command signal to rotate the drive wheels ofthe motion machine in the forward direction as well as to set(correspond with “GO”) the steering wheel in the straight forwarddirection, and is sent to the motion machine. The result is that thetarget machine moves forward.

FIG. 17 is a diagram to show a second example to correlate the voicesignal with operation signal of the machine operation, and correspondsto a case of controlling an air conditioner. The correlation of thevoice signal and the machine operation signal in controlling the motionmachine is as shown in the columns in FIG. 17.

In FIG. 17, if it is desired to cool using the air conditioner, forexample, “COOL” is pronounced as the voice signal 1. And, as the machineoperation signal to correspond with this command, a natural languagecharacter string “C, O, O, L” or, when using binary signals, (“1”, “0”)may be set.

The synthesized sound is emitted from the speaker and is collected bythe microphone at the receiver side. Concurrently, a person in thevicinity hears the machine sound “COOL”, and can readily understand thefact that a command has been issued from the sending apparatus to themotion machine to start cooling action of the air conditioner. At thereceiving side, the following operations are performed.

That is, the operation signal (signal 2) is extracted from thesynthesized sound electrical signal received by the extraction apparatus10. The meaning of the operation signal thus extracted is interpreted bythe machine control section. In this example, it is converted to themachine operation command signal to operate the cooling apparatus in theair conditioner, and is sent to the air conditioner. The result is thatthe air conditioner begins cooling operation.

Here, when the structure shown in FIG. 12 is to be adopted, transmissionof the machine operation signal is carried out directly using thetransmission path 224 without propagating through the air space betweenthe speaker and the microphone. The transmission path may utilize asignal transmission path for normal analogue and digital signals or anoptical link.

FIG. 18 is a diagram to show still another example of the structure ofthe synthesizing apparatus. In the diagram, numeric symbol 1 representsan audible sound signal, 2 a signal, 3 a synthesized sound electricalsignal, 4 a synthesizing apparatus, 226 a voice recognition section, 227a character string/operation signal conversion section, and 228 anatural language character string.

In the example shown in FIG. 18, the features are that the audible soundsignal 1 is a natural voice sound input and that the signal 2 is asignal resulting from voice recognition of the input natural voicesound.

The natural voice input sound is recognized in the voice recognitionsection 226, and is output as a natural language character string 228.This natural language character string 228 is input in the characterstring/operation signal conversion section 227 and is converted to themachine operation signal (signal 2). Then, the natural voice sound(audible sound signal 1) and the machine operation signal (signal 2) aresynthesized in the synthesizing apparatus 4, and a synthesized soundelectrical signal 3 is output.

FIG. 19 is a diagram to show still another example of the synthesizingapparatus. In this diagram, numeric symbol 218 represents an ASCII codetable, 219 an ASCII code converter, and other numeric symbols are thesame as those in FIG. 18.

In FIG. 19, the natural language character string 228 output from thevoice recognition section 226 is input in the ASCII code converter 219.The ASCII code converter 219 converts the natural language characterstring 228 to an ASCII code by referencing the ASCII code table 218.

The signal comprised by the ASCII code is input as signal 2 in thesynthesizing apparatus 4. The synthesizing apparatus 4 synthesizes thissignal with the audible sound signal 1 (natural voice sound) to generatea synthesized sound electrical signal 3, which is output. Otherstructures and operations related to FIGS. 18, 19 are the same as thoseexplained earlier, and explanations are omitted.

Here, in the explanation for Embodiment 3, the example related to thecase of converting the natural language character string to the ASCIIcode, but this is not limited to the ASCII code, and it is obvious thatother character codes can also be used.

Embodiment 4

In Embodiment 4, the purpose is to provide information transmissionmeans that can be operated reliably without being affected by noise,even when the external noise is extremely high, so as not to causeerrors in the transmitted information to trigger erroneous operation ofthe machine.

FIG. 20 shows the basic structure used in Embodiment 4.

In FIG. 20, numeric symbol 1 represents an audible sound signal, 1-1˜1-3are channels, 2 a signal, 3 a synthesized sound electrical signal,304-1˜304-3 amplifiers, 305-1˜305-3 speakers, 306 is a synthesizingapparatus, 306-1˜306-3 synthesizing sections for respective channels,307 a receiving apparatus, 308 an extraction apparatus, 309 a machine,310 a transmission path, and 311 a person.

In FIG. 20, each audible sound signal in each of the channels 1-1˜1-3 issynthesized with signal 2 in the respective synthesizing sections306-1˜306-3 of the synthesizing apparatus 306 to produce respectivesynthesized sound electrical signals 3. The synthesized sound electricalsignal 3 is amplified in the amplifiers 304-1˜304-3 and is output fromthe speakers 305-1˜305-3 as acoustic sound. The person 311 is able tohear the sound.

Further, the synthesized sound electrical signal 3 is sent to the signaltransmission path 310. The receiving apparatus 307 receives thesynthesized sound electrical signal 3 and extracts signal 2 therefrom,and operates the machine 309 according to the signal 2.

In present embodiment, as described earlier, the synthesized soundelectrical signal 3 is amplified in the amplifiers 304-1˜304-3 and isoutput from the speakers 305-1˜305-3 as an acoustic sound, butpre-amplification audible sound signals in respective channels may beamplified and output from the speakers.

However, as in present embodiment, by outputting after amplifying thesynthesized sound electrical signal 3 in the amplifiers 304-1˜304-3, theacoustic output of the synthesized sound electrical signal may bereceived in a microphone and the like, and it is possible to extractsignal 2 from the output of the microphone, and therefore, the range ofapplicability is increased.

FIG. 21 is a diagram to show a first structure of the synthesizingapparatus. In the diagram, 1-1˜1-5 represent channels, 3 represents asynthesized sound electrical signal, 3-1˜3-5 synthesized soundelectrical signals in respective channels, 312-1˜312-5 low-pass filters(abbreviated as LPF in the diagram), 313-1˜313-5 changeover switches(abbreviated as S1˜S5 in the diagram), and 314 a signal 2/switchoperation signal converter.

The signal 2/switch operation signal converter 314 in FIG. 21, uponreceiving a signal 2, converts this signal 2 to a switch operationsignal, and this is given to the changeover switches 313-1˜313-5(S1-S5). The changeover switches 313-1˜313-5 receiving the switchoperation signal perform switching operations according topre-determined settings.

FIG. 22A, 22B show a relationship of signal 2 to the operations of thechangeover switches S1˜S5. In this example, as shown in FIG. 22A, whensending “0” using the synthesized sound electrical signal, thechangeover switches S1, S3, S5 are connected to the lowpass filter side,and when sending “1” using the synthesized sound electrical signal, thechangeover swatches S2, S4 are connected to the lowpass filter side.FIG. 22B shows the relationship of the signal 2 in this condition to theoperation of the changeover switches.

FIG. 23 is a diagram to show an example of the structure of thesynthesizing apparatus. In FIG. 23, numeric symbol 3 represents asynthesized sound electrical signal, 3-1˜3-5 represent synthesized soundelectrical signals in each channel, 315-1˜315-5 highpass filters(abbreviated as HPF in the diagram), 316-1˜316-5 energy detectors,317-1˜317-5 threshold processing section, and 318 a majority logicsection. As explained earlier, on the synthesizing apparatus side shownin FIG. 21, depending on the value “0” or “1” of signal 2, control isexercised whether to output an audible sound signal by passing itthrough the lowpass filter or to output as it is. Thus, by passing thesynthesized sound electrical signal through the highpass filter andanalyzing the frequencies, it is possible to discriminate whether thesynthesized sound electrical signal has been put through the lowpassfilter on the synthesizing apparatus side or has been output as it is.

In other words, when a frequency component higher than a specificfrequency is contained in a signal, this signal has been output from thesynthesizing apparatus side as it is without putting it through thelowpass filter. And, because it has been decided whether to pass or notpass through the lowpass filter for individual channels, depending onwhether the value of signal 2 is “0” or “1”, it is possible todiscriminate whether signal 2 is “0” or “1” from the abovediscrimination result.

Here, the cutoff frequencies of the highpass filters 315-1˜315-5 may beset at approximately the same values as those of the lowpass filters312-1˜312-5 on the synthesizing apparatus side.

The operation will be further explained with reference to FIG. 23. Thesynthesized sound electrical signals 3-1˜3-5 in individual channels areput through respective highpass filters 315-1˜315-5 and are input in theenergy detectors 316-1˜316-5.

In the energy detectors 316-1˜316-5, frequency components are analyzedand the results are determined in the threshold processing section317-1˜317-5. The output from the threshold processing sections317-1˜317-5 are input in the majority logic section 318, and themajority logic section 318 performs majority decision for each output ofthe threshold processing sections 317-1˜317-5, and after determiningwhether signal 2 is “0” or “1”, the results are output.

In present embodiment, because the majority discrimination is performedon the signals embedded in a plurality of channels, the results are lessprone to be affected by the noise, so that signal 2 can be transmittedmore reliably, and their polarities can be determined more accurately.

FIG. 24 is a diagram to show a second example of the synthesizingapparatus. In FIG. 24, numeric symbols 321-1, 321-2 represent audiblesound signals, 3 represents a synthesized sound electrical signal,322-1, 322-2 lowpass filters (abbreviated as LPF in the diagram), 323-1,323-2 highpass filters (abbreviated as HPF in the diagram), 324, 326-1,326-2 are mixers (abbreviated as MIX in the diagram), and 327 anamplifier control signal generation section.

In the example shown in FIG. 24, the audible sound signals 321-1, 321-2,respectively, represent a left channel signal and a right channel signalof a stereo sound. In this diagram, the left channel signal 321-1 thatpassed through the highpass filter 223-1 and the right channel signal321-2 that passed through the highpass filter 323-2 are synthesized inthe mixer 324, and are further input in the amplifiers 325-1, 325-2.

The output from the amplifiers 325-1 is synthesized by the mixer 326-1with the left channel signal 321-1 that passed through the lowpassfilter 322-1, and becomes a left E signal. On the other hand, the outputfrom the amplifiers 325-2 is synthesized by the mixer 326-2 with theright channel signal 321-2 that passed through the lowpass filter 322-2,and becomes a right E signal. Then, a synthesized sound electricalsignal 3 is formed by the left E signal and the right E signal.

The amplifier control signal generation section 327, depending on thevalue “0” or “1” of signal 2, controls the gain of the amplifiers 325-1,325-2. For example, when signal 2 is “0”, the gain on the amplifier325-1 side is controlled so as to be 20 dB lower than normal. On theother hand, when signal 2 is “1”, the gain on the amplifier 325-2 sideis controlled so as to 20 dB lower than normal.

FIG. 25 is a diagram to show a second example of the structure of thesynthesizing apparatus, and this synthesizing apparatus extracts signal2 from the synthesized sound electrical signal output from thesynthesizing apparatus shown in FIG. 24. In FIG. 25, numeric symbol228-1, 228-2 represent highpass filters, 229 represents a comparator.The cutoff frequencies of the highpass filters 228-1˜228-2 may be set atapproximately the same values as those of the highpass filters 223-1,223-2 on the synthesizing apparatus side.

The output from the highpass filters 228-1, 228-2 are compared in thecomparator 229, and are determined as “0” or “1” based on the magnitude,and are output as signal 2.

As described earlier, in the synthesizing apparatus side, it iscontrolled so that, when signal 2 is “0”, the gain of the amplifier225-1 is lower than normal by 20 dB, and when signal 2 is “1”, the gainof the amplifier 225-2 is lower than normal by 20 dB.

Therefore, if the output from the highpass filter 228-1 in FIG. 25 ishigher than the output from the highpass filter 228-2, then signal 2 is“1”, and conversely, if the output from the highpass filter 228-2 ishigher than the output of the highpass filter 228-1, then, signal 2 is“0”. The output signal 2 (either “0” or “1”) is given to the controlsection of the machine as a command signal, and is converted in thecontrol section to a machine operation control signal to operate themachine.

As explained above, in present embodiment, the sending side embedssignals in a plurality of independent channels, and the receiving sideextracts embedded signals by comparing the signals in the plurality ofchannels, and therefore, binary signals can be transmitted at a higherstability and precision.

Embodiment 5

In Embodiment 5, the present invention is applied to a method ofcontrolling the operation of a robot.

For example, there have been toys robots that can dance to music.Specific examples include a toy robot called “dancing flower” which hasan artificial flower or a doll that responds to a music and displaysswing motion.

This toy is constructed so that music and the like is collected by amicrophone housed inside the artificial flower or doll, and is convertedto electrical signals so that the artificial flower or the doll are madeto sway by driving an actuator inside the dancing flower in accordancewith the amplitude of the signals. For this reason, such robotic toyscan only perform repetitive simple motion according to the soundpressure levels without relating to melody or rhythm of the music.

To elevate the dancing motion of a robot to an artistic level, it isnecessary to control its motion in such a way to be congruent with themusical texture represented by melody and rhythm of the music, that is,to choreograph the dancing motion.

In present embodiment, as described below, by embedding motion signalsin the dance music for the robot, detailed movements of the robot can becontrolled using a simple control structure without the need forinformation transmission means for customized motion signal.

FIGS. 26A, 26B show the fundamental structure of the present invention.FIG. 26A shows the sending side and FIG. 26B shows the receiving side.

In FIG. 26A, numeric symbol 401 represents a tone signal and 402represents a dance operation signal, and other numeric symbols are thesame as those shown in FIG. 1A. Also, in FIG. 26B, numeric symbol 411represents a robot control section, 412 an actuator command signal, 413a human-shaped robot, 414 an operational mechanism of the robot, andother numeric symbols are the same as those shown in FIG. 1B.

In the structural diagram of the receiving side shown in FIG. 26B,although the microphone 9, amplifier 5, extraction apparatus 10 androbot control section 411 are drawn separately from the human-shapedrobot 13, it is obvious that such constituting sections on the receivingside may be housed inside the body of the human-shaped robot 13, or theymay be housed inside a platform (not shown) that the human-shaped robot13 is mounted.

Next, the overall flow of the signal will be explained. The sending sidein FIG. 26A is constructed by replacing the audible sound signal 1 inFIG. 1A with the musical signal 40 and signal 2 in FIG. 1A with thedance operation signal 2. Therefore, to the stage of extracting thedance operation signal 402 using the synthesizing apparatus 10 in FIG.26B, it is the same as those in FIGS. 1A and 1B.

Next, the dance operation signal 402 is input in the robot controlsection 411 to generate internally an actuator command signal 412 (orcommand value) for the robot, and the human-shaped robot 413 operates inaccordance with the command signal or the command value. Person in thevicinity can observe the manner of robot motion with the peripheralvision while listening to the dance music to enjoy the dance performanceof the robot.

Next, a method of correlating the music with the movement will beexplained. This is achieved by defining the relative chronologicalrelationship of the tone signal 401 to the dance operation signal 402.As an example, a case to be explained relates to the human-shaped robotshown in FIG. 26B performing a dance operation routine using a musicexpressed by the musical score shown in FIG. 27 to produce a danceroutine by combining four dance operation patterns shown in FIG. 28.

Because the rhythm of this music is four beats, four dance steps are tobe defined per one bar. Designating the dance operation codecorresponding to the j-th beat of the i-th bar by Cij, the flow of thedance operation code of this music, that is, the code sequence can beexpressed as, starting from the first bar: C₁₁ C₁₂ C₁₃ C₁₄ C₂₁ C₂₂ C₂₃C₂₄ . . . .

Next, the dance stance of the robot, that is, as an example of thedancing pattern, four stances A, B, C, D shown in FIG. 28 may beconsidered. That is, A is a stance with both arms up, B is a stance withboth arms down, C is a stance with the right hand up and the left handdown, and D is a stance with the left hand up and the right hand down.

In the following, to simplify the explanation, an example of the motionduring the second bar will be explained. For example, if it is desiredto dance in the order of D, C, B, A, the dance operation codes should bearranged as follows.C₂₁=D, C₂₂=D, C₂₃=B, C₂₄=A  (1)

This dance operation code, as the dance operation code 402, issynthesized with the tone signal 401 in the synthesizing apparatus 4.For this purpose, any of the synthesizing methods explained in theforgoing embodiments can be used.

At the receiving side, dance operation codes that are the same asequation (1) are extracted by the extraction apparatus 10, and the robotcontrol section 411 receives these codes successively, and sendsactuator command values to correspond to the code to the human-shapedrobot 413.

Here, when the tempo of the music is relatively fast, due to delay inrobot control operation or in the transmission delay, it may appearsometimes that the robot motion lags the music. In such a case, thecomposite sounds should be generated so that the overall code sequenceof the dance operation is increased to match the tempo of the music. Byso doing, at the receiver side, at a certain point in time of musicflow, the intended operational pattern code corresponding to thatcertain point in time has already been transmitted to the controlsection of the robot, so that the dance routine may be totallysynchronized to the music without causing the problem of time delay.

FIG. 29 is a diagram to show an example of the structure of thesynthesizing apparatus 4. In FIG. 29, numeric symbol 401 represents atone signal (in this case, music), 402 a dance operation signal (in thiscase, dance operation code), 3 a synthesized sound; 415 a musicaloperation compiling section, and 416 a code embedding section.

In FIG. 29, music 1 and dance operation code 2 are compiled in the musicoperation compiling section 415, and are synthesized in the codeembedding section 416, and are output as a synthesized sound electricalsignal 3. In the following, the operation of the synthesizing apparatuswill be explained using a case of 4/4 beat.

Designating N as the total bar number, mi(t) as an i-th bar tone signal,Pij(t) as an i-th bar j-th beat tone signal, and representing thetime-series by symbols [;], [{ }], the original tone signal waveformMorg can be expressed as follows:Morg(t)={m 1 (t); m 2 (t); . . . ; m N(t)}  (2)where mi(t)={Pi 1 (t); Pi 2 (t); Pi 3 (t); Pi 4 (t)}.

And, the vector of the dance operation signal can be expressed asfollows:Cdance=(C 1 , C 1 . . . C N)  (3)where Ci=(C 11 , C 12 , C 13 , C 14 ).

Further, the tone signal waveform Mhyper synthesized on the basis of thetone signal waveform Morg and the vectors Cdance of the dance operationsignal can be expressed as in equation (4) below:Mhyper(t)={m hyper(1)(t); . . . m hyper (N)(t)}  (4)

The synthesized signal waveform of Cij to Pij is expressed using “h” asequation (5):Pij※Cij

The term [m hyper (1)(t)] in equation (4) can be expressed as equation(6):m hyper(1)(t)={P 11 ※C 11 ; P 12 ※C 12 ; P 13 ※C 13 ; P 14 ※C 14 }  (6)

The generation method of equation (5) is realized by the code embeddingsection 416 shown in FIG. 29. This generation method uses itemsexplained in the foregoing example.

The processing steps for obtaining the tone signal waveform Mhyper(t)shown in equation (4) from the Morg(t) shown in equation (2) and thedance operation signal waveform Cdance shown in equation (3) are asshown in the flowchart in FIG. 30. The designations (S-1)˜(S-9) in thediagram represent processing steps, and the steps S-1˜S-9 are referencedin the following.

(step S-1), first, i is set to the initial value 1.

-   -   (step S-2), next, j is set to the initial value 1.

(step S-3), a synthesized signal waveform of Cij to Pij is generated.

(step S-4), then, it is examined whether j has reached 4.

(step S-S), if j is less than 4, “1” is added to j and return to (stepS-3).

(step S-6), next, it is examined whether i has reached N.

(step S-7), if i is less than N, “1” is added to i and return to (stepS-2).

(step S-8), Mhyper(t) is output, and the process is finished.

Or, in step S-8, instead of outputting collected Mhyper(t), it ispossible to output mhyper(t) for every bar, and by so doing, signalembedding and signal sending can be performed in real time.

FIG. 31 is a diagram to show an example of the structure at thereceiving side in the present embodiment. The flow of signals along thenumeric symbols 7, 9, 5, 3, 10, 402 in FIG. 31 is the same as thatexplained in FIG. 26B.

Also, in FIG. 31, 419 represents a dance operation code/joint angleconversion section, 402 a music tempo obtaining section, 422-1 a θ1target value signal, 422-2 a θ2 target value signal, 423-1 a rotationangle servo mechanism for motor 1, 423-2 a rotation angle servomechanism for motor 2, 424-1 a motor 1, 424-2 a motor 2; 425-1 and 425-2motor drive current, 413 a human-shaped robot, 414 an operation sectionfor the robot, 426-1 a right arm angle (θ1) of the human-shaped robot413, 426-2 a left arm angle (θ2) of the human-shaped robot 413.

In FIG. 31, the synthesized sound from the sending side is collected bythe microphone 9, amplified in the amplifier 5, and its output, which isthe synthesized sound electrical signal 3, is input in the extractionapparatus 10. And, the synthesized sound electrical signal 3 is alsoinput in the musical tempo obtaining section 420. The extractionapparatus 10 extracts a dance operation code 402 from the synthesizedsound electrical signal 3 and outputs the dance operation code 402.

The musical tempo obtaining section 420 generates a music tempo signal421 from the synthesized sound electrical signal 3 and outputs the musictempo signal 421. The dance operation code/joint angle conversionsection 419 generates a θ1 target value signal 422-1 and a θ2 targetvalue signal 422-2, and outputs the signals synchronously with the musictempo signal 421.

The angle θ1 is the right arm angle 426-1 of the human-shaped robot 413,and the angle θ2 is the left arm angle 426-2 of the human-shaped robot413. The rotation angle servo mechanism 423-1 of motor 1 receives thetarget value signal 422-1 of angle 01 and controls rotation angle ofmotor 1 (424-1). The rotation angle servo mechanism 423-2 for motor 2receives the target value signal 422-2 of angle θ2 and controls rotationangle for motor 2 (424-2).

The operation section 414 is a mechanical operational section comprisedmainly by the rotation angle servo mechanism 423-1 for motor 1, therotation angle servo mechanism 423-2 for motor 2, motor 1 (424-1) andmotor 2 (424-2).

Summarizing, the present embodiment related to a case of sending asynthesized sound from the sending side and receiving in the microphoneon the receiving side, and extracting the operation signal for thehuman-shaped robot from the synthesized sound electrical signal, but itis obvious that the synthesized sound electrical signal may be sent fromthe sending side to the receiving side through a wired transmissionpath, and the receiving side then extracts the operation signal from thesynthesized sound electrical signal to operate the human-shaped robot.

Embodiment 6

In Embodiment 6, road information and other information necessary fordriving are provided to a navigation apparatus installed in anautomobile.

Drivers of automobiles would like to obtain, while driving, variousfresh external information such as traffic information, weatherforecast, sightseeing information and commercial information reliably,safely, simply and quickly at low cost. In the past, information fordrivers has been provided through normal radio and televisionbroadcasting. Also, in recent years in Japan, information providingservices based on so-called “visible radio” through FM multiplexed textbroadcasting, VICS information and car navigation system and others havealso been realized.

Of such information obtaining means described above, radio broadcastingenables information to be received readily inside the vehicle using alow cost facility so that it will continue to be used in the future.However, the driver is able only to listen to the content of radiobroadcasting, and because the driver is not able to take notes duringdriving, it is difficult to retain records of important information.

Therefore, present embodiment provides an economical means for providingvoice information to car drivers through radio broadcast waves and dataother than voice data concurrently.

FIGS. 32A, 32B are block diagrams of an example of the basic structureof the present embodiment, and FIG. 32A shows the sending side and FIG.32B shows the receiving side.

In FIG. 32A, numeric symbol 501 represents original sending sound(audible sound signal), 502 represents information for car navigationand the like different than original sending sound 501, 3 a synthesizedsound electrical signal, 4 a synthesizing apparatus, 505 a sendingapparatus, and 506 a sending antennae.

In FIG. 32B, numeric symbol 507 represents a receiving apparatus, 508 areceiving antennae, 509 an amplifier (abbreviated as AMP in thediagram), 10 an extraction apparatus, 511 a speaker, 512 a synthesizedsound, 513 a person (driver), and 514 a car navigation apparatus, andother numeric symbols are the same as those in FIG. 32A.

An outline of the flow of the signal in FIGS. 32A, 32B will beexplained. First, on the sending side, the original sending sound 501and the information 502 to be transmitted are electrically synthesizedusing data hiding technique, for example, under a condition such thatthe auditory presence of information 502 cannot be sensed by the person,and a radio signal modulated by the synthesized sound electrical signal3 is emitted from the sending apparatus 505 through the antennae 650into the air space. At the receiving side (automobile), the emitted waveis captured by the receiving antennae 508, demodulated in the receivingapparatus 507 to extract the synthesized sound electrical signal 3. Thissynthesized sound electrical signal 3 is amplified in the amplifier 509and is output from the speaker 511 as a voice sound (synthesized sound)512. The driver 513 is able to obtain voice information according to thesynthesized sound 512.

On the other hand, the synthesized sound electrical signal 3, which isthe detected wave by the receiving apparatus 507, is input in theextraction apparatus 10, and information 502 is extracted from thesynthesized sound electrical signal 3 by the extraction apparatus 10.The extracted information 502 is input in the car navigation apparatus514 to perform desired operations such as display and the like such asdisplaying information on the display section of the car navigationapparatus 514 or providing information to the control program of the carnavigation apparatus.

Generation of the synthesized sound electrical signal 3 in thesynthesizing apparatus 4 is performed using the same method explained inthe embodiments. Also, the hardware structures of the synthesizingapparatus 4 and extraction apparatus 10 may be the same as thoseexplained in the embodiments above.

FIG. 33 is a block diagram to show an example of the structure of theapparatus in the receiving side (automobile side). In FIG. 33, numericsymbol 507 represents a receiving apparatus, 508 a receiving antennae,509 an amplifier (abbreviated as AMP in the diagram), 10 an extractionapparatus, 511 a speaker, 512 a synthesized sound, and 513 a person(driver).

Also, 515 represents an extraction timing switching apparatus, 516 aninformation storage apparatus, 517 a target district informationdatabase, 518 a route generation apparatus, 519 an input/outputapparatus, 520 an extraction timing signal, and 521 a and 521 brepresent operation signals input by the driver.

Overall flow of the signal will be explained with reference to FIG. 33.

The person (driver) 513 is driving while listening to radio broadcastingthrough a car radio in the vehicle. When information of interest isbroadcast, by performing certain operations, an operation signal 521 ais generated to inform the extraction timing switching apparatus 515. Inresponse, the extraction timing switching apparatus 515 generates anextraction timing signal 520.

Then, the extraction apparatus 10 starts the process of extractinginformation 502, and information 502 is automatically stored in theinformation storing apparatus 516. Then, later on, when the driverdesires, by generating an operation signal 521 b given by the driver'sinstruction, information stored in the information storing apparatus 516is registered in the target district information database 517. Theinformation thus registered in the target district information database517 is able to be used as target district information by the carnavigation apparatus.

The input/output apparatus 519 displays map data recorded on a mediumsuch as CD (compact disc) and DVD (digital versatile disc) on a displayapparatus (not shown), and, the route generation apparatus 518 generatespathways from the current location to the target destination location,and displays it on the display apparatus by superimposing on the map.

In the following, some specific examples of providing commercialinformation using the present apparatus will be explained. In thisexample, as the original broadcast sound 501, a commercial message for arestaurant is heard. The commercial message conveys the following voiceof an announcer along with the background music.

[This is ABC restaurant located in Tokyo, Musashino city, Midoricho,telephone number is 0422-59-0000, Internet address is www.abc.cojp].Information 502 related to the original broadcast sound 501 includes thename of the restaurant, its longitude and latitude, telephone number,and URL (uniform resource locator), which are arranged in the text datasequence, for example, ABC restaurant, E135N30, 0422-59-0000,www.abc.co.jp”. Such information 2 is superimposed on the originalbroadcast sound 1 using the data hiding technique.

Broadcasting station broadcasts such a synthetic sound as normal sound.In the meantime, the driver is listening to this broadcasting, and whenthe driver decides that it is of interest, reception button (not shown)is turned on and an operation signal 521 a is generated. Also, anextraction timing signal 520 is generated by this operation. When theextraction apparatus 10 recognizes the operation signal 520, theinformation 502 is extracted as text data.

The extracted information 502 is stored in the information storingapparatus 516, and the driver can use this automatically registeredrestaurant information at any desired time. For example, the driver canphone the restaurant for reservation, and may set this restaurant as thedestination in the car navigation apparatus.

The route generation apparatus 518 in the car navigation apparatusgenerates route information on the basis of the longitude-latitudeinformation contained in signal 2 that was extracted earlier and thecurrent longitude-latitude information, and guides the driver to thesite by displaying the route on the map.

Also, in the above embodiment, the timing for extraction operation forinformation 2 is when the driver listening to the broadcast decides thatit is of interest and generates an operation signal 521 a by operating abutton, however, it is permissible not to adopt this approach, and, thestructure may be arranged, for example, so that when radio broadcastingcontaining the information 2 begins, extraction of information 2 andstoring action in the information storing apparatus 516 may beautomatically started, and if it is found later that it is of nointerest the driver can delete the registration.

Also, in the explanation provided for the above embodiment, although abeneficial effect is obtained by inputting the information embedded inthe voice signal broadcast in the car navigation apparatus of anautomobile, resulting in increasing the utility of the car navigationapparatus, utilization method of the present invention is not limited tosuch an approach, and it is obvious that there are many other methods ofutilization such as transmitting emergency information as information 2,or blinking the display lamp, or displaying words on the displayapparatus.

Also, in the explanation provided for the above embodiment, thesynthesized sound electrical signal is transmitted by broadcasting, butit is clear that, instead of broadcasting to target a large number oflisteners simultaneously, the invention can be applied to transceiversto communicate on a one-on-one basis.

Further, it is obvious that similar system may be applied not only forautomotive devices but also to general household information devices.That is, by embedding various information necessary for operation ofhousehold electrical appliances in the sound source contained intelevision and radio broadcasting received by households, users are ableto conveniently obtain information at low cost, enabling to operatehousehold electrical appliance conveniently.

Embodiment 7

Next, Embodiment 7 of the present invention will be explained. Presentembodiment enables to fuse broadcasting received in households andcommunication through the Internet.

Presently, virtually every household has a radio receiver that canreceive middle frequency broadcasting or short-wave broadcasting or FMbroadcasting. Also, in recent years, increasing number of householdshave appropriate environment to enable connection to the Internet.

Also, radio receiver is provided not only in households but also inautomobiles in most cases to enable to listen to broadcasting. Also,there has been a rapid increase in the number of cars that are equippedwith television receiver and car navigation system.

Also, opportunity is increasing for using portable personal computerinside the automobile.

Due to such recent widespread use of the Internet, program contents ofconventional radio and TV, information related to programs andinformation to supplement the programs are often published through theInternet web page, and the listeners/viewers are now able to listen/viewbroadcast programs with reference to such web pages. With this trend,URL of homepage is often broadcast during broadcasting. Also, URLnotification is also given routinely by commercial broadcasts.

However, when URL is publicized during programs and advertisementsduring radio and TV broadcasting, the listener/viewer interested in suchinformation needs to take notes of such URL, and refer to the note tooperate a keyboard to input the URL in PC, resulting in a cumbersomeprocess.

Also, it may also be considered to transmit text information such as URLthrough text multiplexing broadcasting, or so-called “visible radio”,but this method presents a problem that it requires broadcast station tonewly install expensive facilities and the receiver side must also havededicated receiver.

Further, information providing services attached to car navigationsystems include Internet connection service, information portal serviceoffered by car navigation system companies and providing informationservices based on conversation with operators; however, because contactsare made through public communication means, such as portable phones, inall cases, there are problems of cumbersome process and the need forservice fees.

The present embodiment enables to transmit voice sounds to users(receiving side) through broadcast waves as well as data other than thevoice sounds, for example, to send URL data to be extracted at thereceiving side, and, using a PC, rapidly access homepage of this URL onthe Internet.

FIGS. 34A, 34B are block diagrams of the structure of the presentembodiment, and FIG. 34A shows the sending side and FIG. 34B shows thereceiving side.

In FIG. 34A, numeric symbol 601 represents an original broadcastingsound (audible sound signal), 602 a network address expressed in binaryformat different than the original broadcast sound 601, 3 a synthesizedsound electrical signal, 4 a synthesizing apparatus, 605 a sendingapparatus, 606 a sending antennae, and 607 a broadcasting station. Also,in FIG. 34B, numeric symbol 608 represents a receiving antennae, 609 areceiving apparatus, 610 a speaker, 611 a sound collection apparatus(microphone), 612 a PC, 613 an extraction apparatus, 614 a controlapparatus, 615 a NW access apparatus (browser), 616 a display apparatus,and 617 a network.

Overall flow of the signal in FIGS. 34A, 34B will be explained. First,at the sending side (broadcasting station), using a data hidingtechnique for example, the broadcasting sound 601 and the networkaddress 602 to be transmitted are synthesized electrically underconditions that a person is not able to detect the auditory presence ofthe network address 602, and the synthesized sound is input in thesending apparatus 605 as the synthesized sound electrical signal 3. Inthe sending apparatus 605, radio carrier wave is modulated with thesynthesized sound electrical signal 3 and it is emitted from the sendingantennae 606 as the sending electrical wave into the air space.

At the receiving side, this sending electrical wave is captured by thereceiving antennae 608, is demodulated in the receiving apparatus 609,and the synthesized sound electrical signal 3 is reproduced. Then, thesynthesized sound electrical signal is input in the speaker 610, andconverted to an acoustic signal, which is output. The person is notaware that the acoustic output contains network address 602 and listensto the sound as the original broadcast sound 1.

The acoustic output is collected by the sound collection apparatus 611,is converted to the synthesized sound electrical signal and is input ina personal computer 612. The personal computer 612, using the extractionapparatus 613, successively extracts binary signals from the synthesizedsound electrical signal, and reproduces the network address 602.

This network address 602 is forwarded to the network access apparatus(browser) 615 through the control apparatus 614, and the network accessapparatus 615 accesses the network 617 and obtains information from thelocation corresponding to the network address 602 (homepage and thelike). Then, the network access apparatus 615 displays the obtainedinformation on a display apparatus 616 through the control apparatus614.

FIG. 35 is a block diagram to show another example of the structure ofthe embodiment. In FIG. 35, 618 represents a TV receiver, 620 theInternet, 621 a program sponsor, and 622 a viewer (PC user), and othernumeric symbols are the same as those shown in FIGS. 34A, 34B.

In FIG. 35, at the broadcasting station 607, responding to request ofthe program sponsor 621, within the voice sounds of the TV program beingbroadcast or within the commercial message, the address of the homepageof the program sponsor established on the Internet using thesound-watermark technology.

As a specific example, a case of an XYZ company promoting an xyz productwill be explained. The XYZ Company provides original commercial soundsfor the xyz product to the broadcasting station. Within the commercialmessage, the homepage address of the XYZ Company, for example,w/w.xyz.co.jp is embedded as a binary string, using the sound-watermarktechnology.

The broadcasting station 607 broadcasts this synthesized sound in amanner similar to the images and normal commercial message. The viewer622 viewing TV while operating the PC 612 listens to this commercialmessage and if the xyz product is of interest, when the viewer presses acertain key on the keyboard of the personal computer 612, thesynthesized sound flowing from TV is converted to the synthesized soundelectrical signal through the microphone, and is input in the extractionapparatus of the personal computer.

Then, the homepage address, www.xyz.co.jp, of the XYZ Company isextracted from the synthesized sound as a binary string. At this time,the extracted binary strings are successively accumulated, and each byteis decoded as a numeral or a word, and the network address (for example,the homepage address www.xyz.co.jp) is reproduced, and the pagecorresponding to this address is displayed by the browser on the displayapparatus 616.

Based on the homepage address, automatically or manually, the personalcomputer 612 accesses the homepage address on the Internet. Then, theuser can order the product xyz through the EC (electronic commerce) siteof the homepage. The user's ears hear the synthesized sound as normalsound, so that, to those users who are not interested in the service,the commercial message sounds normal.

The frequency utilization band in present embodiment is the same as thatexplained using FIG. 2. Also, the structure of the synthesizingapparatus may be the same as that explained in FIG. 4. Also, theprinciple of operation of the synthesizing apparatus is the same as thatexplained using FIGS. 4A, 4B. Also, the structure of the extractionapparatus may be the same as that explained using FIG. 5. Also, theoperational procedures of the synthesizing apparatus and extractionapparatus is the same as those explained in Embodiment 1.

It can be easily surmised from the explanations provided above relatingto Embodiment 7 that the operation of the main steps related to thepresent invention can be realized by microprocessors in the personalcomputer executing relevant programs. Especially, the receiver side inthe embodiment is based on personal computers so that their mainoperations can be readily realized by a program executed by thecomputer.

For such application programs, at the sending side, original broadcastsound (601) and a binary network address (602) different than theoriginal sound are synthesized electrically to generate a synthesizedsound electrical signal for modulation, and the modulated radio signalis received by the receiving apparatus, analyzed to demodulate thesynthesized sound electrical signal, and generate an acoustic signalthrough the speaker, and this is collected by the sound collectionapparatus to convert into the synthesized sound electrical signal, whichbecomes an input signal.

In this program, a step of successively extracting the binary signalshidden in the synthesized sound electrical signal, a step of reproducinga network address based on the binary signals, a step of obtaininginformation by accessing the homepage on the Internet using the networkaddress, and a step of outputting the obtained information are describedas an executable program, and this program is stored in an executableform in the memory section of the personal computer. Then, this programis booted as necessary to perform the above steps.

Such a program is pre-recorded in a computer-readable recording medium.

Embodiment 8

Embodiment 8 provides an additional function by fusion of the controlmethod for a robot present in Embodiment 5 and broadcasting orcommunication.

In other words, the purpose of the embodiment is not only to transmitvoice directly to the robot, but also to operate the robot or togenerate voices from the robot by using the aspects of TV broadcastingto transmit signals through the voice output of a TV receiver. Further,an interactive system is to be realized between the viewer of TV and thebroadcasting station to feedback through the robot to transmit theviewer's intentions to the station side.

FIG. 36 is a diagram to show an outline of the present embodiment, whichshows an example of transmitting synthesized sound through TVbroadcasting. In FIG. 36, numeric symbol 730 represents a TV receiver,731 a TV screen, 732 a face of an actor on the TV screen, 733 a voicesound output from the TV receiver, 735 a robot, 736 a voice output fromthe robot, 737 an operation of the robot.

FIG. 37 is a block diagram of the functions of the robot shown in FIG.36, and in FIG. 37, numeric symbol 741 represents a microphone, 742 aninformation extraction apparatus, 743 a motion control apparatus, 744 arobot operation command correlation table, 745 a driving mechanism, 746a voice control apparatus, 747 a robot speech command correlation table,and 748 a speaker.

In the following, the present embodiment will be explained in detailusing FIGS. 36, 37. The voice sound 733 output from the TV receiver isthe voice of the actor appearing on the TV show, and it is “How are you”in a natural language. The voice sound 733 output from the TV receiveris, in fact, a synthesized sound, and within the natural language “Howare you”, a control signal for the robot is embedded in the actor'soriginal voice (audible sound signal).

And, information regarding words and motion that appearing to respond tothe voice sound 733 of the actor's words output from the TV receiver isembedded as the control signal in the audible sound signal to betransmit to the robot. In this example, the embedded control signals areword information to convey “I am fine” and motion information to “wavethe right hand”.

FIG. 38 shows an example of the data for the robot operation commandcorrelation table 744. And, FIG. 39 shows an example of the data for therobot speech command correlation table 747. In other words, operationsand speeches to correspond to such commands are pre-stored as a table inthe robot. Here, “A” and “B” in FIG. 38 are command codes for the robot,and operations to correspond to each command are defined in the table.Similarly, “0” and “1” in FIG. 39 are also command codes for the robot,and speeches to correspond to the command codes are defined in thetable.

When it is desired to make the robot to vocalize “I am fine” and performa motion to “wave the right hand”, commands corresponding to “A” and “0”are embedded at the broadcasting station side in the voice sound “Howare you” of the actor, and are broadcast. The receiving side receivesthis in the TV receiver 730, and outputs the voice sound of the actor“How are you”.

The robot 735 receives this voice output in the microphone 741, which isoutput as the synthesized sound electrical signal. The informationextraction apparatus 742 extracts a binary signal embedded in thesynthesized sound electrical signal, and forwards it to the motioncontrol apparatus 743 and the voice control apparatus 746.

In the motion control apparatus 743, if “A” or “B” symbol is detected inthe binary code, it searches in the robot operation command correlationtable 744 according to the symbol to find the content of the operation(in this case, “wave the right hand”), and a signal corresponding tothis operation is handed to the drive mechanism 745. The drive mechanism745 carries out the operation of “wave the right hand” according to thesignal.

Also, in the voice control apparatus 746, if “0” or “1” symbol isdetected from the extracted binary signal, it searches in the robotspeech command correlation table 747 to find the content of the voice tobe pronounced, and synthesizes the relevant voice and outputs voicesignal from the speaker.

As another example, if it is desired to make the robot wave the lefthand along with the speech of “I am fine”, the control signal should beset to “B” and “0”.

In the above explanation, the robot speech command correlation table(747) is pre-registered in the robot and used, however, separate fromthis, without using such a correlation table, embedding the speed textsas part of the robot control signal may be considered. In the aboveexample, (“A”, “I am fine”) may be used as the robot control signal. Inthis case, the robot extracts a text named “I am fine”, and vocalize “Iam fine” by voice synthesis.

FIG. 40 shows an example of realizing interactive TV broadcasting usingthe robot in the present embodiment. In FIG. 40, numeric symbols 730,731, 732, 735, 737 as the same as those in FIG. 36. Also, 734 representsvoice output from the TV receiver, 738 a voice output from the robot,739 a and 739 b are touch sensors.

FIG. 41 is a block diagram to show the functional structure of therobot. In FIG. 41, numeric symbols 741˜748 are the same as those inFIGS. 37, and 749 represents an input apparatus, 750 a controlapparatus, and 751 a communication control apparatus.

FIG. 42 is a block diagram to show the relationship of broadcasting tonetwork to realize a communication-broadcasting convergence system, andnumeric symbol 752 represents a robot, 753 a network, 754 acommunication control apparatus, 755 a summing apparatus, 756 a displayapparatus, 757 a player, 758 a broadcasting facility, and 759 a TVreceiver.

In the following, the present example will be explained with referenceto FIGS. 40˜42. In this example, the voice sound broadcast on the TV isthe voice of the actor appearing on the TV, and represents a naturallanguage that means “Those in favor, press the right hand, thoseopposing, press the left hand”. A control signal correspond-ing to thewords of the actor is embedded as the information 702 in the voicesound, and this information 702 is transmitted to the robot to producedesired motion and words. In this case, information 702 contains theword “which” and the operation information related to waving both hands.

FIG. 43 shows data for a robot operation command correlation table 744.Also, FIG. 44 shows data for a robot speech command correlation table747. The data in the correlation tables are pre-loaded in the robot. Asshown in FIG. 40, to let the robot pronounce the word “which” and towave both hands, information 702 should be set to (“A”, “0”).

As another example, if it is desired to let the robot to say “how areyou” and to perform a welcome, information 702 should be set to (“B”,“1”).

In the example shown above, data in the robot speech command correlationtable (747) is pre-registered in the robot, but, separate from such amethod, speech texts may be embedded directly as information 702. Whenembedding speech texts, information 702 may be made such that (“A”,“DOCCHI”). In this case, the robot is made to extract a text “DOCCHI”and to pronounce “which” by voice synthesis.

After which, the robot is readied to accept commands in the touch sensor39 a or 39 b provided in each hand. These touch sensors correspond tothe input apparatus 749 in the case of the function block diagram inFIG. 41. The viewer, according to the contents of programming, pressesleft hand touch sensor 39 a or right hand touch sensor 39 b.

The input from the input apparatus 749 in FIG. 41 is correlated to apart of the information output from the information extraction apparatus742 and is sent to the network 753 by way of the communication controlapparatus 751.

Here, information to discriminate such as network address (telephonenumber and the like), information to identify the broadcasting station,information to identify programs, and information to identify the wordsof the players are contained in a part of the information output fromthe information extraction apparatus 742. That is, it is the informationto correlate the viewer input command to the exact portion of thebroadcast program.

As shown in FIG. 42, information input and sent from a large number ofusers is collected in the collection apparatus 757 by way of the network753, and the results are displayed on the display apparatus 756. Theplayers 757 in the TV program and the person in charge view the numberspresented on the display apparatus 756 to carry out communication withthe users.

Embodiment 9

Next, Embodiment 9 will be explained with reference to FIGS. 45˜51.Embodiment 9 relates to the case of controlling a robot through apersonal computer.

FIG. 45 is a diagram to show the overall structure of the roboticsystem, in which a personal computer (robot control signal generationapparatus) 801 and a robot shaped as an animal such as a bear 802 areconnected by an audio line (electrical signal input means) 803. Therobot 802 is configured such that the robot can nod using an internalservo motor (servo motor 826 in FIG. 47), and can emit sounds into theair space from an internal speaker (speaker 822 in FIG. 47). In thisrobotic system, the synthesized sound electrical signal and operationinformation are transmitted from personal computer 801 to robot 802through the audio line 803.

FIG. 46 is a function block diagram on the sending side, that is, thepersonal computer 801 side. The personal computer 801 is comprised by arobot control section (robot operation signal generation means) 811, asynthetic sound generation section (audible sound signal generationmeans) 812, a motor control section 813, an AM modulator 814, an adder(electrical signal generation means) 815, and a sound board (electricalsignal generation means) 816.

FIG. 47 is a function block diagram on the receiving side, that is, therobot 802 side. The robot 802 is comprised by an amplifying section 821,a speaker (emitting means) 822, a bandpass filter (separation extractionmeans) 823, an AM demodulator 824, a motor drive section 825, and aservo motor (driving means) 826.

The flow of signal will be explained with reference to FIGS. 46 and 47.First, in the personal computer 801, which is the sending side, therobot control section 811 generates text information S801 and, from thegenerated text information S801, a motor rotation angle command valueS802 for the servo motor 826, as a robot operation signal, is generated.As an example, it is also possible for the robot control section 811 tobe constructed so that text information S801 and motor rotation anglecommand value for the servo motor 816 are generated from HTML (hypertextmarkup language) files, and electronic mail texts by performingmorphological analysis and the like.

Technology for generating text information S801 and motor rotation anglecommand value S802 using the morphological analysis technique will beexplained briefly.

First, morphological analysis is a technique for separating text filessuch as electronic mail into individual words (i.e., a morpheme) inorder to provide a tag for each word, in order to analyze the contentsof the text according to the morphological analysis and to obtain taginformation related to emotions. An actual case of morphologicalanalysis will be explained using the following sentence. <Input mailtext example>“Excuse me, this is Misae. This is an invitation to a homeparty at Sachan's house. This time, let's make it a pot-luck party.”

From these sentences, words to suggest a tag (abbreviated as cue-word)are extracted and emotion tag information is obtained. In the case ofthe above sentences, from words “excuse me” an emotion tag [greetings]is obtained, and from words “home party” an emotion tag [enjoyment] isobtained, and from the last words “pot-luck” an emotion tag [invitation]is obtained.

Then, the cue-words for emotion tag information are stored in the speechdatabase as voice sound files, that have been modulated with tonalparameters (pitch frequency, power, resonance) to provide linguistictonality, so that when the input mail text is output, it is possible tooutput it as a speech pattern with some emotional content.

Here, cue-words to be associated with emotion tag information may beconsidered to include the following. Cue-words for an emotion tag[happiness] may include “success” and “congratulations” and the like,cue-words for an emotion tag [anger] may include “complaints”, “anger”and the like, cue-words for an emotion tag [sadness] may include“failure”, “pain” and the like, cue-words for an emotion tag [pleasure]may include “laughter”, “enjoyment” and the like, cue-words for anemotion tag [apology] may include “sorry”, “forgive me” and the like,cue-words for an emotion tag [surprise] may include ‘eh?’, “whoa” andthe like, cue-words for an emotion tag [emphasis] may include “! ”,“urgent” and the like.

Then, in the present embodiment, to operate the robot in accordance withthe emotion tag obtained by the morphological analysis, by correlatingto the emotion tag information, command values (motor number, location,speed, time) for the robot joint drive motor (servo motor 826 in thepresent embodiment) are arranged chronologically beforehand, and theresults are stored in the gesture database. By so doing, operationsignals for the robot are generated in reference to the jester databaseaccording to the emotion tag information obtained by using themorphological analysis.

FIG. 48 is a function block diagram to show the details of the robotcontrol section 811 in the present embodiment. In the robot controlsection 811, the morphological analysis section 841 performsmorphological analysis on the text information to extract cue-words, andthe emotion tag information extraction section 842 obtains emotion taginformation on the basis of the cue-words. The robot operation signalgeneration section 843 generates robot operation signals on the basis ofthe emotion tag information obtain in reference to gesture database 844.

Here, returning to FIG. 46, the synthetic sound generation section 812generates a synthesized voice sound S803 based on the text informationS801 sent from the robot control section 811. Such a technology is knownto be disclosed in, for example, in Nobuhiko Kitawaki editor, “SpeechCommunication Technology—Speech and Acoustics Technologies forMultimedia Services”, pp. 62–86, Corona, 1996 (reference 5).

Also, the motor control section 813 generates a motor control signalS804 from the motor rotation angle command value S802 sent from therobot control section 811. This motor control signal S804 is a valuederived by a simple first-order equation from the motor rotation anglecommand value S802. Then, the motor control signal S804 isamplitude-modulated in the AM modulator 814 to generate an AM modulatedwave S805. Here, the carrier wave for the AM modulated wave S805 may beselected in the vicinity of 18 KHz so that the AM modulated wave wouldnot be heard by human ears.

The AM modulated wave S805 and the synthesized sound signal S803 aresynthesized in the adder 815, and are sent to the sound board 816 in thepersonal computer 801, and are sent to the audio line 803 as asynthesized sound electrical signal S806.

The synthesized sound electrical signal S806 sent from the personalcomputer 801 passes through the audio line 803 and is input in theamplifying section 821 and the bandpass filter 823 of the robot 802,which is the receiving side. In the robot 802, the synthesized soundelectrical signal S806 is amplified using the amplifier 821, and isemitted into the air space. Therefore, the person is able to hear thevoice sound emitted from the speaker 822.

In the meantime, the bandpass filter 823 having its center value at theAM carrier wavelength extracts the AM modulated wave from thesynthesized sound electrical signal S806, and the AM modulator 824reproduces the motor control signal S807 from the motor control signalS807. From the motor control signal S807, the motor drive section 825generates a servo motor command signal S808 for operating the servomotor 826. The servo motor 826, based on the servo motor command signalS808, rotates while controlling the rotation angle at the value presetby the sending side (personal computer 801) and makes the robot 802 tooscillate its neck.

FIG. 49 shows an example of the relation, in the present embodiment, ofthe text information S801 from the sending side, synthesized soundsignal S803, motor rotation angle command value S802, motor controlsignal S804, AM modulated wave S805, and the synthesized soundelectrical signal S806. In this example, it is imagined that the robot802 announces “mail from Kaori has arrived”, and tilts its neck fromleft to right.

Here, if it is desired for the robot to have a number of degrees offreedom, as shown in FIG. 50, in the sending side (personal computer801), a plurality of motor control sections 813, AM modulators 814, andadders 815 are readied, and the AM frequency of the carrier wave shouldbe varied slightly. Also, the receiving side should prepare in the samemanner, and a plurality of bandpass filters 823, AM modulators 824,motor drive sections 825 and servo motors 826 should be readied.

As explained above, according to the present embodiment of the roboticsystem, synthesized sound signal S803 and motor control signal S804 canbe transmitted at the same time from the personal computer 801 to therobot 802 using only the audio line 803. Also, without providing aspecial structure, emission of sound from the speaker 822 and the neckoscillation operation of the robot 802 by the servo motor 826 can becarried out synchronously. Therefore, the robotic system can besimplified and the cost lowered. Also, because the signal line foroperating the robot does not need to be dedicated in the personalcomputer 801, while the robot 802 is being operated, scanner, terminaladapter, printer and the like can also be operated simultaneously.

FIG. 51 shows a flowchart of the process of electrically synthesizing asound signal. In the present embodiment, an application program forexecuting the steps in this flowchart is recorded on a hard disc(recording medium) of the personal computer 801. Such a program can alsobe recorded on other computer-readable recording medium such as floppydisc, opto-magnetic disc, ROM, CD-ROM and the like.

First, in step S101, an audible sound signal from text information isgenerated, and in step S102, a robot operation signal is generated fromthe text information. Next, in step S103, a synthesized sound electricalsignal is generated by synthesizing the audible sound signal generatedin step S101 and the robot operation signal generated in step S102.Next, progressing to step S104, the synthesized sound electrical signalsynthesize in step S103 is output. Here, in this flowchart, step S101and step S102 are concurrently processed, but step S101 may be carriedout first followed by step S102. Conversely, step S102 may be carriedout followed by step S101.

Embodiment 10

Embodiment 10 will be explained with reference to FIGS. 52 to 54. Here,in Embodiment 10, the parts that are the same as those in Embodiment 9are given the same numeric symbols in FIGS. 52 to 54.

The overall structure of the robotic system is the same as that inEmbodiment 9, and is as shown in FIG. 45.

FIG. 52 is a function block diagram of the sending side, which is thepersonal computer 801 side. The personal computer 801 is comprised by arobot control section (robot operation signal generation means) 811, asynthetic sound generation section (audible sound signal generationmeans) 812, a motor control section 813, an AM modulator 817, an adder(electrical signal generation means) 815, and a sound board (electricalsignal generation means) 816.

FIG. 53 is a function block diagram of the receiving side, that is, therobot 802 side. The robot 802 is comprised by an amplifier section 821,a speaker (emitting means) 822, a bandpass filter (separation extractionmeans) 827, a frequency counter 828, a motor drive section 825, a servomotor (driving means) 826, and a timer circuit 829.

The flow of signal will be explained with reference to FIGS. 52 and 53.First, in the personal computer 801, which is the sending side, therobot control section 811 generates text information S801 and, from thegenerated text information S801, a motor rotation angle command valueS802 for the servo motor 826, as a robot operation signal, is generated.It is permissible to adopt the method of generating the text informationS801 and the motor rotation angle command value S802 as in Embodiment 9.

The synthetic sound generation section 812 generates a synthesized voicesound S803 based on the text information S801 sent from the robotcontrol section 811. It is permissible to adopt the same method ofgenerating the synthesized voice sound S803 as that used in Embodiment9.

The motor control section 813 generates a motor control signal S804 fromthe motor rotation angle command value S802 sent from the robot controlsection 811. This motor control signal S804 is a value derived by asimple first-order equation from the motor rotation angle command valueS802. Then, the motor control signal S804 is FM-modulated in the FMmodulator 817 to generate an FM modulated wave S809. Here, the carrierwave for the FM modulated wave S809 may be selected in the vicinity of18 KHz so that the FM modulated wave S809 would not be heard by humanears.

The FM modulated wave S809 and the synthesized sound signal S803 aresynthesized in the adder 815, and are sent to the sound board 816 in thepersonal computer 801, and are sent to the audio line 803 as asynthesized sound electrical signal S806.

The synthesized sound electrical signal S806 sent from the personalcomputer 801 passes through the audio line 803 and is input in theamplifying section 821 and the bandpass filter 823 of the robot 802 asthe receiving side. In the robot 802, the synthesized sound electricalsignal S806 is amplified using the amplifier 821, and is emitted intothe air space. Therefore, the person is able to hear the voice soundemitted from the speaker 822.

In the meantime, the bandpass filter 827 having its center value at theAM carrier wavelength extracts the FM modulated wave from thesynthesized sound electrical signal S806, and the frequency counter 828measures the frequency of the FM modulates wave. The motor drivingsection 825 generates a servo motor command signal S808 to drive theservo motor 826 according to the measured value of the frequency counter828. The motor drive section 825 generates a servo motor command signalS808 for operating the servo motor 826. The servo motor 826, based onthe servo motor command signal S808, rotates while controlling therotation angle at the angle preset by the sending side (personalcomputer 801) and makes the robot 802 to oscillate its neck. Here, bythe action of the timer circuit 829, the frequency measurement and servomotor command signal S808 are generated periodically.

FIG. 54, in Embodiment 10, is a diagram to show an example of therelation of sending side information text information S801, synthesizedsound signal S803, motor rotation angle command value S802, FM modulatedwave S809, synthesized sound electrical signal 806. In this example, itis imagined that the robot 802 announces “mail from Kaori has arrived”,and tilts its neck from left to right. As can be understood from thisdiagram, in the present embodiment, the system is arranged so that thehigher the motor rotation angle command value S802 the higher thefrequency of the FM modulated wave S809.

Here, if it is desired for the robot to have a number of degrees offreedom, as shown in FIG. 50 in Embodiment 9, in the sending side(personal computer 801), a plurality of motor control sections 813, FMmodulators 817, and adders 815 are readied, and the FM frequency of thecarrier wave should be varied slightly. Also, the receiving side shouldprepare in the same manner, and a plurality of bandpass filters 827,frequency counters 828, motor drive sections 825 and servo motors 826should be readied.

According to the present embodiment of the robotic system, synthesizedsound signal S803 and motor control signal S804 can be transmitted atthe same time from the personal computer 801 to the robot 802 using onlythe audio line 803.

The same effects as those in Embodiment 9 can be obtained by the roboticsystem in Embodiment 10. That is, according to the robotic system inEmbodiment 10, using only the audio line 803, synthesized sound signalS803 and motor control signal S804 can be transmitted at the same timefrom the personal computer 801 to the robot 802. Also, without providinga special structure, emission of sound from the speaker 822 and the neckoscillation of the robot 802 by the servo motor 826 can be carried outsynchronously. Therefore, the robotic system can be simplified and thecost lowered. Also, because the signal line for operating the robot doesnot need to be dedicated in the personal computer 801, while the robot802 is being operated, scanner, terminal adapter, printer and the likecan also be operated simultaneously.

Also, in the robotic system in Embodiment 10, although the apparatusbecomes somewhat more complex compared with the robotic system inEmbodiment 9, compared with the system using the AM modulator inEmbodiment 9, more stable operation is possible and the acoustic effectsare improved.

Embodiment 11

Next, Embodiment 11 will be explained with reference to the diagram inFIG. 55.

FIG. 55 is a system configuration diagram of a robotic system inEmbodiment 11, and difference between this embodiment and Embodiments 9and 10 is that in Embodiments 9 and 10, the synthesized sound electricalsignal synthesized by synthesizing the synthesized sound signal androbot operation signal is transmitted from the personal computer 801 tothe robot 802 through the audio line 803, in Embodiment 11, however, thesynthesized sound electrical signal is transmitted through a radiotransmission apparatus 804.

In detail, the personal computer 801 is connected to the transmitter 804a of the radio transmission apparatus 804 through the audio line 803 a,and the robot 802 is connected to the receiver 804 b of the radiotransmission apparatus 804, and the synthesized sound electrical signalgenerated in the personal computer 801 is transmitted to the robot 802by way of the radio transmission apparatus 804. In the robotic system inEmbodiment 11 also, the robot 802 amplifies the transmitted synthesizedsound signal and emits it into the air space, and the robot operationsignal is demodulated to oscillate the neck. Here, in Embodiment 11, theaudio line 803 b and the receiver 804 b of the radio transmissionapparatus 804 constitute the electrical signal input means of the robot802.

For the radio transmission apparatus 804, apparatus based on infrared orelectrical waves may be considered. Also, in the robotic system inEmbodiment 11, either the AM modulation method described in Embodiment 9or the FM modulation method described in Embodiment 11 may be utilized.In the robotic system in Embodiment 11, the advantage is that physicaltransmission line is not necessary between the personal computer 801 andthe robot 802.

Embodiment 12

Embodiment 12 will be explained with reference to the diagrams in FIGS.56 and 57.

FIG. 56 is a configurational diagram of the robotic system in Embodiment12, and in Embodiment 12, there is no personal computer 801, a compactdisc player (abbreviated as CD player below) 805 and a robot 802 areconnected by an audio line 803. Here, the robot 802 is the same as therobot 802 in Embodiment 9 or 10.

And, in the robotic systems described in Embodiments 9, 10, thesynthesized sound electrical signal is generated in the personalcomputer 801, but in Embodiment 12, the synthesized sound electricalsignal is written in the compact disc (abbreviated as CD below) 806.Then, when this CD 806 is played in the CD player 805 to read thecontents of CD 806, the synthesized sound electrical signal synthesizedby the synthesized sound and the robot operation signal is sent from theCD player 805 to the robot 802 and the robot emits sounds from thespeaker and oscillates its neck.

Therefore, in this case also, it is possible to link the sound andmotion of the robot 802. Also, in the robotic system in Embodiment 12,the advantage is that the personal computer 801 is not required.

FIG. 57, in the robotic system in Embodiment 12, is a diagram to show anexample of the method of writing the synthesized sound electrical signalin a CD. The example of writing the synthesized sound electrical signalin the CD will be explained with reference to FIG. 57.

The structures of the robot control section 811, synthetic soundgeneration section 812, motor control section 813, FM modulator 817,adder 815, are the same as the structures inside the personal computer810 in Embodiment 10, and their explanations are omitted. Thesynthesized sound signal generated in the syntheses sound generationsection 812 and the FM modulated wave modulated by the FM modulator 817serving as the robot operation signal FM are multiplexed in the adder815 and becomes the synthesized sound electrical signal. Thissynthesized sound electrical signal is stored once in the disc apparatus830. The stored data are quantized at 16-bits and are sampled at 44.1KHz.

Sampled data can be duplicated by pressing using a pressing apparatus840 as in normal musical CD to manufacture the CD containing thesynthesized sound electrical signals.

Embodiment 13

Next, Embodiment 13 will be explained with reference to the diagrams inFIGS. 58 and 59.

The overall configuration of the robotic system in Embodiment 13 is thesame as that in Embodiment 9, and will be explained using FIG. 45. Therobotic system is comprised by a personal computer (computer system) 801and the robot 802, which are connected by the audio line (audio signalinput means) 803. The robot 802 is able to oscillate the neck using theservo motor housed internally, and is able to emit sounds into the airspace using the internal speaker. Here, the audio line 803 has an audioline L (left) channel 803L and an audio line R (right) channel 803R.

FIG. 58 is a function block diagram, on the sending side, that is, thepersonal computer 801 side. The personal computer 801 is comprised by avoice-music control section (audible sound generation means) 831, amotor control section 832, a modulator (audio signal generation means)833, and a stereo audio output apparatus (audio signal outputting means)834. The stereo audio outputting apparatus 834 has two channels, anL-channel and an R-channel, for outputting audio signals.

FIG. 59 is a function block diagram, on the receiving side, that is, therobot 802 side. The robot 802 is comprised by a speaker (broadcastingmeans) 835, a demodulator (robot operation signal reproducing means)836, a motor drive section 837, and a servo motor (drive means) 838.

Next, the flow of signal will be explained with reference to FIGS. 58and 59. First, the personal computer 801, which is the sending side, thevoice-music control section 831 generates an audible sound signal ofvoice or music, and outputs this audible sound signal to the stereoaudio control section 834 as well as to the motor control section 832.The motor control section 832 generates a motor rotation angle commandsignal from the audible sound signal input from the voice-music controlsection 831, and outputs it to the modulator 833. The modulator 833modulates or encodes the motor rotation angle command signal input fromthe motor control section 832 to generate an audio signal, and output itto the stereo audio outputting apparatus 834. Here, the method ofgenerating audio signal in the modulator 833 may include various methodssuch as AM modulation, FM modulation, PM modulation, PC modulation andthe like. The carrier frequency of the modulated wave may be anyfrequency, but it is preferable to choose frequency regions that areinaudible to human ears.

Then, the stereo audio outputting apparatus 834 sends the audible soundsignal input from the voice-music control section 831 to the robot 802from the audio signal outputting L-channel through the audio lineL-channel 803L, and at the same time, sends the audio signal input fromthe modulator 833 for driving the motor to the robot 802 from the audiosignal outputting R-channel through the audio line R-channel 803R.

The robot 802, which is the receiving side, emits the audible soundsignal input by way of the audio line L-channel 803 into the air spacethrough the speaker 835. Therefore, the person is able to hear the soundor music emitted from the speaker 835.

In the meantime, the audio signal input through the audio line R-channel803R is input in the demodulator 836, and the demodulator 836demodulates or decodes the audio signal to reproduce the motor rotationangle command signal, and output it to the motor drive section 837. Themotor drive section 837 generates a servo motor command signal to drivethe servo motor, and outputs it to the servo motor 838. Thus, the servomotor 838 rotates to the rotation angle preset by the sending side(personal computer 801) according to the servo motor command signal, andoscillates the neck of the robot 802.

As explained above, according to the robotic system in Embodiment 13, itis possible to communicate between the personal computer 801 and therobot 802 by transmitting the audible sound signal through the audioline L-channel 803L and the robot operation signal through the audioline R-channel 803R, so that there is no need for a dedicated operationsignal line. Therefore, the robotic system can be simplified and thecost lowered. Also, because the signal line for operating the robot doesnot need to be dedicated in the personal computer 801, while the robot802 is being operated, scanner, terminal adapter, printer and the likecan also be operated simultaneously.

Also, in Embodiments 9˜11, an audible sound signal and a robot operationsignal are synthesized in the personal computer 801 to generate asynthesized sound electrical signal, and this synthesized soundelectrical signal is transmitted to the robot 802 through the audio line803 and others, and in robot 802, the robot operation signal isseparated and extracted from the received synthesized sound electricalsignal, and using the extracted robot operation signal the servo motor826 is driven, so that it is necessary to provide synthesizing means forsynthesizing the synthesized sound electrical signal and separatingextracting means for separating and extracting the robot operationsignal from the synthesized sound electrical signal, but in Embodiment13, the audible sound signal and the robot operation signal arecommunicated independently so that the synthesizing means or separatingextracting means become unnecessary, and the system is simplified.

Embodiment 14

In Embodiment 14 to be explained next, the present invention is appliedto a system for advertisement and sales promotion.

FIG. 60 is a block diagram to show the structure of the system inEmbodiment 14. This system, originated by a retailer's request orthrough a service offered by an advertising agent, embeds couponinformation in a commercial message broadcast by voice only, or acommercial message broadcast by voice and image by a broadcastingstation using the data hiding technique and the like, and the consumerreceiving this message obtains an incentive such as discount from theretailer by accumulating points from the coupons.

As shown in FIG. 60, advertising agent has a synthesizing apparatus 904that can embed digital information other than the voice sound in thevoice sound, using the data hiding technique and the like. Also, thisadvertising agent is provided with a web server 921 and a consumerdatabase 922.

Also, the broadcasting station has a program sending apparatus 905 and atransmitter 906.

Also, the consumer has a receiving antennae 908 for receiving broadcastssent from the sending antennae 907, a receiver 909, an extractionapparatus 910 for extracting information embedded in the voice sound,and a personal computer 911.

Also, the retailer has a terminal for receiving and viewing theinformation that is accumulated in the consumer database 922.

Next, the flow of the signal in the present embodiment will beexplained. The retailer produces a commercial message 901 foradvertising own company, and request an advertising agent to embedcoupon information in the commercial message 901, and broadcast thisembedded commercial message. Upon receiving the request, the advertisingagent side produces coupon data 902 for the commercial message, andusing a synthesizing apparatus 904, embeds a signal (digital signal) 2based on coupon data 902 in the audible sound signal 1 in the commercialmessage 901, and outputs it as the synthesized sound electrical signal3. Here, because the synthesizing apparatus 904 synthesizes using thedata hiding technique, a listener cannot recognize signal 2 in thesynthesized sound output from the synthesizing apparatus 904.

Coupon data 902 contains, at least, sending discrimination informationfor identifying dispatching of the commercial message, sending date andtime, effective period of the coupon, and coupon point information.

The synthesized sound electrical signal 3 is sent out according tobroadcasting schedule by the program sending apparatus 905 in thebroadcasting station, modulated in the transmitter 906, and is sent outfrom the sending antennae 907 as electrical waves. At the consumer side,based on the voice sound received in the receiving antennae 908 anddemodulated in the receiver 909, signal 2 is extracted by the extractionapparatus 910, and the extracted signal is forwarded to the personalcomputer 911. The personal computer is provided with dedicated software,and the coupon data 902 is reproduced by executing this software, andthe coupon points are accumulated.

As the points are accumulated, the consumer can receive an incentiveequal to the saved points by accessing the web server 921. In this case,communication between the personal computer 911 and the web server 921is performed through the Internet, for example. The web server 921receives the coupon data accumulated by this consumer from the consumerside, and after checking its legitimacy by matching with the originalcoupon data 902, writes the point data of this consumer in the consumerdatabase. Here, because the web server 921 checks the sendingdiscrimination information contained in the coupon data 902, when theconsumer side makes an request to accumulate points using a non-existentsending discrimination information or to accumulate points more than twotimes for the same sending discrimination information, such requests canbe denied.

Accordingly, information on the points written in the consumer database922 and an effective period of a point can be inspected using a terminal923 provided in the retailer, and based on this, the retailer awards theconsumer a special incentive such as price discount.

According to the present embodiment, because the consumer can not onlyenjoy the process of accumulating the points for obtaining incentivesbut the consumer also becomes actively interested in searching forbroadcast commercial messages, so that the effect of advertising isincreased, leading to enhanced sales.

Here, the present embodiment relates to business transactions betweenretailer and consumer, but the present invention is not limited to this,it is possible to utilize the sales promotional system to all types ofbusiness transactions among any parties.

The processes described above may be performed by recording applicationprograms for performing the processes, and loading and executing theprograms in a computer system to synthesize signals, transmit thesynthesized signals, extract the signal from the synthesized sound, andto exercise various types of controls using the extracted signal.“Computer system”, in this context, includes any OS (operating systems)and peripheral hardwares.

Computer-readable recording media include portable media such as floppydisks, opto-magnetic disks, ROM, CD-ROM, as well as fixed memory devicessuch as hard disks housed in computer systems. Further,computer-readable recording media include those that hold programs for agiven length of time, such as volatile memories (RAM) containedinternally in computers for servers and client computer systems used intransmitting application programs through networks such as the Internetor communication lines of telephone circuits.

The above programs may be transmitted from the computer system storingsuch programs in a memory device and the like to other computer systemsthrough a transmission medium or by transmission waves through thetransmission medium. Here, a transmission medium for transmittingprograms refer to a medium having a capability to transmit informationsuch as networks (communication networks) represented by the Internet orcommunication lines represented by telephone circuits.

Application programs may perform a part of the described functions.Further, they may be operated in conjunction with pre-recorded programsalready stored in computer systems to provide differential files(differential programs).

So far, various embodiments of the present invention have been describedwith reference to the diagrams, but the specific structures are notlimited to those exemplified, and includes designs that can be includedwithin a range of the essence of the present invention.

1. An apparatus for transmitting acoustic signals comprising: asynthesizing apparatus for electrically synthesizing an audible soundsignal and an insertion signal, and outputting a synthesized soundelectrical signal; an acoustic signal outputting apparatus forconverting the synthesized sound electrical signal and outputting theacoustic signal externally; an extraction apparatus for extracting saidinsertion signal based on the synthesized sound electrical signal outputfrom the synthesizing apparatus; a sound collection apparatus forreceiving an acoustic signal output from the acoustic signal outputtingapparatus and re-converting to a synthesized sound electrical signal,the extraction apparatus extracting said insertion signal using thesynthesized sound electrical signal converted by the sound collectionapparatus, said insertion signal being a code information correspondingto a character string in a natural language; a machine control sectionfor controlling said machine according to the signal extracted by theextraction apparatus; and a sound generation machine for pronouncing thecharacter string in a natural language corresponding to the codeinformation extracted according to command of the machine controlsection.
 2. An apparatus for transmitting acoustic signals comprising: asynthesizing apparatus for electrically synthesizing an audible soundsignal and an insertion signal, and outputting a synthesized soundelectrical signal; an acoustic signal outputting apparatus forconverting the synthesized sound electrical signal and outputting theacoustic signal externally; an extraction apparatus for extracting saidinsertion signal based on the synthesized sound electrical signal outputfrom the synthesizing apparatus; a sending apparatus for sending asynthesized sound electrical signal output from the synthesizingapparatus; a receiving apparatus for receiving the synthesized soundelectrical signal sent by the sending apparatus; wherein the extractionapparatus extracts said insertion signal using the synthesized soundelectrical signal received by the receiving apparatus, said insertionsignal being a code information corresponding to a character string in anatural language; a machine control section for controlling said machineaccording to the signal extracted by the extraction apparatus; and asound generation machine for pronouncing the character string in anatural language corresponding to the code information extractedaccording to command of the machine control section.
 3. A method fortransmitting acoustic signals comprising: a synthesizing step forsynthesizing an audible sound signal and an insertion signal to generatea synthesized sound electrical signal; an acoustic signal outputtingstep for converting the synthesized sound electrical signal to anacoustic signal and outputting the acoustic sound externally; atransmitting step for transmitting the synthesized sound electricalsignal; an extracting step for extracting said insertion signal from thesynthesized sound electrical signal that has been transmitted; and amachine control step for controlling a machine according to the signalextracted in the extracting step; wherein said transmitting step relatesto sending the synthesized sound electrical signal using a sendingapparatus at the sending side, and the signal is received at thereceiving side using a receiving apparatus; wherein in said synthesizingstep, an audible sound is divided into a plurality of channels and saidaudible signal in each channel and said insertion signal are synthesizedindividually to produce a synthesized sound electrical signal in eachchannel; in said acoustic sound outputting step, said synthesized soundelectrical signal in each channel is converted to an acoustic signalwhich is output externally; and in said transmitting step, saidsynthesized sound electrical signal is transmitted for each channelthrough a transmission path; wherein said channel is comprised by twochannels represented by a left channel and a right channel for astereophonic audible sound signal; wherein in said synthesizing step, anaudible sound signal in each channel is input into a lowpass filter anda highpass filter, respectively, and, after controlling output levels ofthe highpass filter in both channels according to the insertion signal,each controlled output is synthesized with a filtered signal processedthrough the lowpass filter to generate a synthesized sound electricalsignal for each channel; and in said extracting step, the synthesizedsound electrical signal in each channel is individually input in thehighpass filter, and the insertion signal is extracted by comparingoutput results from the highpass filters.
 4. An apparatus fortransmitting acoustic signals comprising: a synthesizing apparatus forelectrically synthesizing an audible sound signal and an insertionsignal, and outputting a synthesized sound electrical signal; anacoustic signal outputting apparatus for converting the synthesizedsound electrical signal and outputting the acoustic signal externally;an extraction apparatus for extracting said insertion signal based onthe synthesized sound electrical signal output from the synthesizingapparatus; a sending apparatus for sending a synthesized soundelectrical signal output from the synthesizing apparatus; a receivingapparatus for receiving the synthesized sound electrical signal sent bythe sending apparatus; wherein the extraction apparatus extracts saidinsertion signal using the synthesized sound electrical signal receivedby the receiving apparatus; and a machine control section forcontrolling an operation of a machine using the signal extracted by theextraction apparatus; wherein the synthesizing apparatus is providedwith a plurality of synthesizing sections for individually synthesizingan audible sound signal separated into a plurality of channels and theinsertion signal; the acoustic signal outputting apparatus is providedwith a plurality of acoustic output sections for converting thesynthesized sound electrical signal in each channel into an acousticsignal and exporting externally; and the sending apparatus and thereceiving apparatus receive and transmit synthesized sound electricalsignals in individual channels through transmission paths; wherein saidchannel is comprised by two channels represented by a left channel and aright channel for a stereophonic audible sound signal; and wherein saidsynthesizing apparatus is provided with a lowpass filter and a highpassfilter for inputting an audible sound signal of each channel; a levelcontroller for controlling output levels of both channels of a highpassfilter according to the insertion r signal; and a mixer for mixing asignal output from the level controller and a filtered signal processedthrough a lowpass filter; and said extraction apparatus is provided witha highpass filter for inputting the synthesized sound electrical signalof each channel and a comparator for extracting the insertion signal bycomparing output results from highpass filters.
 5. An apparatus fortransmitting acoustic signals comprising: a synthesizing apparatus forelectrically synthesizing an audible sound signal and an insertionsignal, and outputting a synthesized sound electrical signal; anacoustic signal outputting apparatus for converting the synthesizedsound electrical signal and outputting the acoustic signal externally;an extraction apparatus for extracting said insertion signal based onthe synthesized sound electrical signal output from the synthesizingapparatus; a sending apparatus for sending a synthesized soundelectrical signal output from the synthesizing apparatus; a receivingapparatus for receiving the synthesized sound electrical signal sent bythe sending apparatus; wherein the extraction apparatus extracts saidinsertion signal using the synthesized sound electrical signal receivedby the receiving apparatus; and a machine control section forcontrolling an operation of a machine using the signal extracted by theextraction apparatus; wherein the synthesizing apparatus is providedwith a plurality of synthesizing sections for individually synthesizingan audible sound signal separated into a plurality of channels and theinsertion signal; the acoustic signal outputting apparatus is providedwith a plurality of acoustic output sections for converting thesynthesized sound electrical signal in each channel into an acousticsignal and exporting externally; and the sending apparatus and thereceiving apparatus receive and transmit synthesized sound electricalsignals in individual channels through transmission paths; and whereinsaid audible sound signal is a music signal; said machine to becontrolled is a robot; and said machine control signal is a danceoperation signal for controlling a dance operation of the robot insynchronization with the music signal.
 6. An apparatus for transmittingacoustic signals comprising: a synthesizing apparatus for electricallysynthesizing an audible sound signal and an insertion signal, andoutputting a synthesized sound electrical signal; an acoustic signaloutputting apparatus for converting the synthesized sound electricalsignal and outputting the acoustic signal externally; an extractionapparatus for extracting said insertion signal based on the synthesizedsound electrical signal output from the synthesizing apparatus; asending apparatus for sending a synthesized sound electrical signaloutput from the synthesizing apparatus; a receiving apparatus forreceiving the synthesized sound electrical signal sent by the sendingapparatus; wherein the extraction apparatus extracts said insertionsignal using the synthesized sound electrical signal received by thereceiving apparatus; and a machine control section for controlling anoperation of a machine using the signal extracted by the extractionapparatus; wherein the synthesizing apparatus is provided with aplurality of synthesizing sections for individually synthesizing anaudible sound signal separated into a plurality of channels and theinsertion signal; the acoustic signal outputting apparatus is providedwith a plurality of acoustic output sections for converting thesynthesized sound electrical signal in each channel into an acousticsignal and exporting externally; and the sending apparatus and thereceiving apparatus receive and transmit synthesized sound electricalsignals in individual channels through transmission paths; wherein saidaudible sound signal is a music signal; said machine to be controlled isa robot; and said machine control signal is a dance operation signal forcontrolling a dance operation of the robot in synchronization with themusic signal; and wherein said dance operation signal is a code signalto express a pattern of the dance operation of the robot.
 7. Anapparatus for transmitting acoustic signals comprising: a synthesizingapparatus for electrically synthesizing an audible sound signal and aninsertion signal, and outputting a synthesized sound electrical signal;an acoustic signal outputting apparatus for converting the synthesizedsound electrical signal and outputting the acoustic signal externally;an extraction apparatus for extracting said insertion signal based onthe synthesized sound electrical signal output from the synthesizingapparatus; a sending apparatus for sending a synthesized soundelectrical signal output from the synthesizing apparatus; a receivingapparatus for receiving the synthesized sound electrical signal sent bythe sending apparatus; wherein the extraction apparatus extracts saidinsertion signal using the synthesized sound electrical signal receivedby the receiving apparatus; and a machine control section forcontrolling an operation of a machine using the signal extracted by theextraction apparatus; wherein the synthesizing apparatus is providedwith a plurality of synthesizing sections for individually synthesizingan audible sound signal separated into a plurality of channels and theinsertion signal; the acoustic signal outputting apparatus is providedwith a plurality of acoustic output sections for converting thesynthesized sound electrical signal in each channel into an acousticsignal and exporting externally; and the sending apparatus and thereceiving apparatus receive and transmit synthesized sound electricalsignals in individual channels through transmission paths; wherein saidaudible sound signal is a music signal; said machine to be controlled isa robot; and said machine control signal is a dance operation signal forcontrolling a dance operation of the robot in synchronization with themusic signal; and wherein said synthesizing apparatus, when synthesizinga dance operation signal and an audible sound signal which is the musicsignal, allows said dance operation signal to precede said audible soundsignal.
 8. An apparatus for transmitting acoustic signals, comprising: asynthesizing step for synthesizing an audible sound signal and aninsertion signal electrically to generate a synthesized sound electricalsignal; a modulating step for generating a radio signal by modulatingwith the synthesized sound electrical signal; a transmitting step fortransmitting the radio signal; a demodulating step for receiving theradio signal and demodulating the received signal to generate theaudible sound signal; an acoustic signal outputting step for convertingthe synthesized sound electrical signal into an acoustic signal andoutputting a converted signal; an extracting step for extracting theinsertion signal according to the synthesized sound electrical signaldemodulated in said demodulating step; and a control step forcontrolling an apparatus according to the signal extracted in saidextracting step; wherein said demodulating step and said acoustic signaloutputting step and said extracting step are performed in an apparatusinstalled in an automotive vehicle, and in said control step, a carnavigation apparatus is controlled.
 9. An apparatus for transmittingacoustic signals, comprising: a synthesizing apparatus for electricallysynthesizing an audible sound signal and an insertion signal to generatea synthesized sound electrical signal; a modulating apparatus forgenerating a radio signal by modulating with the synthesized soundelectrical signal; a demodulating apparatus for receiving the radiosignal and demodulating to generate the synthesized sound electricalsignal; an acoustic signal outputting apparatus for converting thesynthesized sound electrical signal into an acoustic signal andoutputting the acoustic signal; an extracting apparatus for extractingthe insertion signal from the synthesized sound electrical signaldemodulated by the demodulating apparatus; and a control apparatus forcontrolling an apparatus by the insertion signal extracted by theextraction apparatus; wherein said demodulating apparatus and saidacoustic signal outputting apparatus and said extraction apparatus areinstalled in an automotive vehicle, and said control apparatus controlsa car navigation apparatus.
 10. An apparatus for transmitting acousticsignals, comprising: a synthesizing apparatus for electricallysynthesizing an audible sound signal and an insertion signal to generatea synthesized sound electrical signal; a modulating apparatus forgenerating a radio signal by modulating with the synthesized soundelectrical signal; a demodulating apparatus for receiving the radiosignal and demodulating to generate the synthesized sound electricalsignal; an acoustic signal outputting apparatus for converting thesynthesized sound electrical signal into an acoustic signal andoutputting the acoustic signal; an extracting apparatus for extractingthe insertion signal from the synthesized sound electrical signaldemodulated by the demodulating apparatus; and a control apparatus forcontrolling an apparatus by the insertion signal extracted by theextraction apparatus; wherein said demodulating apparatus and saidacoustic signal outputting apparatus and said extraction apparatus areinstalled in an automotive vehicle, and said control apparatus controlsa car navigation apparatus; and wherein said audible sound signalrepresents a broadcast content of a program or a commercial message;said insertion includes a name, a telephone number, locationinformation, and information on a uniform resource locator (URL) thatare related to the broadcast content; and said control apparatuscontrols so as to register names, telephone numbers, locationinformation, and information on uniform resource locators (URL) that arerelated to contents of broadcasting in the car navigation apparatus.